Green0318/G.Commune

0

stars

23

commits

Python

primary language

Mar 22, 2024

updated

README

Commune AI

License: MIT Discord Chat Website Uptime Twitter Follow

An Open Modules Network

Convert a Python module that starts a client session and sends requests into Rust

Benefits

  • Performance: Rust is known for its performance and can be much faster than Python due to its emphasis on zero-cost abstractions.

  • Concurrency: Rust's ownership model and safety guarantees allow for safe concurrent programming, making it easier to handle multiple requests efficiently with threads.

Convert python code to rust.

async with aiohttp.ClientSession() as session:
            async with session.post(url, json=request, headers=headers) as response:
            if response.content_type == 'text/event-stream':

            ...

            elif response.content_type == 'text/plain':
                    # PROCESS TEXT EVENTS
                    result = await asyncio.wait_for(response.text(), timeout=timeout)
            else:
                raise ValueError(f"Invalid response content type: {response.content_type}")

How to

Two challenges to be solved to improve performance and support concurrency.

First, how to create rust module and use this in python file.

Created rust module which generates a thread whenever client requests and send http result to each request in that thread.

To import the rust module in the python code, compiled the rust code into a shared library that can be loaded by python.

Second, how to call python module in rust code.

Need to use this python module in the rust code.


if self.debug:
    progress_bar = c.tqdm(desc='MB per Second', position=0)
result = []


let gil = Python::acquire_gil();
let py = gil.python();
// Import the tqdm module and get the tqdm function
let tqdm = py.import("tqdm")?.getattr("tqdm")?;
// Call tqdm function with desired arguments
let progress_bar = tqdm.call1(("MB per Second",0))?; 

...

progress_bar.call_method1("update", (event_bytes / BYTES_PER_MB,))?;

Developement FAQ

  • Where can i find futher documentation? This repository folder, Doc.
  • Can I install on Windows? Yes, Guide.
  • Can I contribute? Absolutely! We are open to all contributions. Please feel free to submit a pull request.

Contributors

goru-tensor

19 commits

Green0318

4 commits

Green0318/G.Commune

0

stars

23

commits

Python

primary language

Mar 22, 2024

updated

README

Commune AI

License: MIT Discord Chat Website Uptime Twitter Follow

An Open Modules Network

Convert a Python module that starts a client session and sends requests into Rust

Benefits

  • Performance: Rust is known for its performance and can be much faster than Python due to its emphasis on zero-cost abstractions.

  • Concurrency: Rust's ownership model and safety guarantees allow for safe concurrent programming, making it easier to handle multiple requests efficiently with threads.

Convert python code to rust.

async with aiohttp.ClientSession() as session:
            async with session.post(url, json=request, headers=headers) as response:
            if response.content_type == 'text/event-stream':

            ...

            elif response.content_type == 'text/plain':
                    # PROCESS TEXT EVENTS
                    result = await asyncio.wait_for(response.text(), timeout=timeout)
            else:
                raise ValueError(f"Invalid response content type: {response.content_type}")

How to

Two challenges to be solved to improve performance and support concurrency.

First, how to create rust module and use this in python file.

Created rust module which generates a thread whenever client requests and send http result to each request in that thread.

To import the rust module in the python code, compiled the rust code into a shared library that can be loaded by python.

Second, how to call python module in rust code.

Need to use this python module in the rust code.


if self.debug:
    progress_bar = c.tqdm(desc='MB per Second', position=0)
result = []


let gil = Python::acquire_gil();
let py = gil.python();
// Import the tqdm module and get the tqdm function
let tqdm = py.import("tqdm")?.getattr("tqdm")?;
// Call tqdm function with desired arguments
let progress_bar = tqdm.call1(("MB per Second",0))?; 

...

progress_bar.call_method1("update", (event_bytes / BYTES_PER_MB,))?;

Developement FAQ

  • Where can i find futher documentation? This repository folder, Doc.
  • Can I install on Windows? Yes, Guide.
  • Can I contribute? Absolutely! We are open to all contributions. Please feel free to submit a pull request.

Contributors

goru-tensor

19 commits

Green0318

4 commits

Languages

Python

64.4%

Solidity

20.8%

Jupyter Notebook

7.8%

Rust

3.1%

JavaScript

2.6%