Shadowgraph
About
Sign in

Tencent/CogKernel

35

stars

0

commits

Python

primary language

Feb 21, 2025

updated

README

Cognitive Kernel

CK

An Early-stage “Autopilot” System for Everyone

Project Page | Paper | Model

Cognitive Kernel is an open-sourced agent system designed to achieve the goal of building general-purpose autopilot systems. It has access to real-time and private information to finish real-world tasks.

In this repo, we release both the system and the backbone model to encourage further research on LLM-driven autopilot systems. Following the guide, everyone should be able to deploy a private 'autopilot' system on their own machines.

Installation

Step 1: Start the model server

  1. Download the model weights from the following resources.
Model NameModel SizeModel Link
llama3_policy_model_8b8Blink
llama3_policy_model_70b70Blink
helper_model_1b1Blink
  1. Start the model servers

  • 2.1 Start the main policy model

    • Step 1: Install the vllm package. 
      pip install vllm==0.5.4
    • Step 2: Launch the vllm service. 
      python -m vllm.entrypoints.openai.api_server --model path/to/downloaded/policy/model --worker-use-ray --tensor-parallel-size 8 --port your_port --host 0.0.0.0 --trust-remote-code --max-model-len 8192 --served-model-name ck

  • 2.2 Start the Helper LLM

    • Step 1: Install the TGI framework.

    • Step 2: Launch the TGI service:

      CUDA_VISIBLE_DEVICES=YOUR_GPU_ID text-generation-launcher --model-id PATH_TO_YOUR_HELPER_LLM_CHECKPOINT --port YOUR_PORT --num-shard 1 --disable-custom-kernels

    • Step 3: Configure service URLs.

      Update the service_url_config.json file. Replace the values for the following keys with the IP address and port of your Helper LLM instance:

      • concept_perspective_generation
      • proposition_generation
      • concept_identification
      • filter_doc
      • dialog_summarization

  • 2.3 Start the text embedding model

    • Step 1: Install the required dependencies:

      Install cherrypy and sentence_transformers.

    • Step 2: Launch the embedding service:

      python text_embed_service.py --model gte_large --gpu YOUR_GPU_ID --port YOUR_PORT --batch_size 128

    • Step 3: Configure service URLs:

      Update the service_url_config.json file. Replace the value of sentence_encoding with the IP address and port of your text embedding service instance.

  1. Edit the configuaration files:

    docker-compose.yml:

    • add the "IP:Port" of the policy model (8b/70b) servers to SERVICE_IP;

Step 2: Start the system

  1. Download and install the Docker desktop/engine

  2. Go to the repo folder and then

docker-compose build

or

docker compose build
  1. Start the whole system with
docker-compose up

or

docker compose up

Then, you should be able to play with the system from your local machine at (http://0.0.0.0:8080). PS: for the first time you start the system, you might observe an error on some machines. This is because the database docker takes some time to initiate. Just kill it and restart, the error should be gone.

System Demonstration

Demo 1: Search for citations of an uploaded paper on Google Scholar.


Demo 2: Download a scientific paper and ask related questions.

Contribution

We welcome and appreciate any contributions and collaborations from the community.

Disclaimer

Cognitive Kernel is not a Tencent product and should only be used for research purpose. Users should use Cognitive kernel carefully and be responsible for any potential consequences.

Citation

If you find Cognitive Kernel is helpful, please consider cite the following technical report.

@article{zhang2024cognitive,
    title={Cognitive Kernel: An Open-source Agent System towards Generalist Autopilots},
    author={Zhang, Hongming and Pan, Xiaoman and Wang, Hongwei and Ma, Kaixin and Yu, Wenhao and Yu, Dong},
    year={2024},
}

Contact

For technical questions or suggestions, please use Github issues or Discussions.

For other issues, please contact us via cognitivekernel AT gmail.com.

Similar repos

CognitiveKernel/helper_1b_v1

Model

0

Tencent/KsanaLLM

549

same org

microsoft/JARVIS

JARVIS, a system to connect LLMs with ML community. Paper: https://arxiv.org/pdf/2303.17580.pdf

25,238

myshell-ai/AIlice

AIlice is a fully autonomous, general-purpose AI agent.

1,417

EmbraceAGI/LocalAGI

LocalAGI：Locally run AGI powered by LLaMA, ChatGLM and more. ｜ 基于 ChatGLM, LLaMA 大模型的本地运行的 AGI

81

AidenFaulconer/AI.DEN

Local agentic LLM using the latest and best in AI (MTP+Quant+Qwen3.5+caveman+claw-code)

2

superlinked/sie

Open-source inference server and production cluster for all the models your agent needs.

3,229

xingyaoww/code-act

Official Repo for ICML 2024 paper "Executable Code Actions Elicit Better LLM Agents" by Xingyao…

1,700

Languages

Python

66.6%

TypeScript

24.2%

JavaScript

8.6%

Tencent/CogKernel

35

stars

0

commits

Python

primary language

Feb 21, 2025

updated

README

Cognitive Kernel

CK

An Early-stage “Autopilot” System for Everyone

Project Page | Paper | Model

Cognitive Kernel is an open-sourced agent system designed to achieve the goal of building general-purpose autopilot systems. It has access to real-time and private information to finish real-world tasks.

In this repo, we release both the system and the backbone model to encourage further research on LLM-driven autopilot systems. Following the guide, everyone should be able to deploy a private 'autopilot' system on their own machines.

Installation

Step 1: Start the model server

  1. Download the model weights from the following resources.
Model NameModel SizeModel Link
llama3_policy_model_8b8Blink
llama3_policy_model_70b70Blink
helper_model_1b1Blink
  1. Start the model servers

  • 2.1 Start the main policy model

    • Step 1: Install the vllm package. 
      pip install vllm==0.5.4
    • Step 2: Launch the vllm service. 
      python -m vllm.entrypoints.openai.api_server --model path/to/downloaded/policy/model --worker-use-ray --tensor-parallel-size 8 --port your_port --host 0.0.0.0 --trust-remote-code --max-model-len 8192 --served-model-name ck

  • 2.2 Start the Helper LLM

    • Step 1: Install the TGI framework.

    • Step 2: Launch the TGI service:

      CUDA_VISIBLE_DEVICES=YOUR_GPU_ID text-generation-launcher --model-id PATH_TO_YOUR_HELPER_LLM_CHECKPOINT --port YOUR_PORT --num-shard 1 --disable-custom-kernels

    • Step 3: Configure service URLs.

      Update the service_url_config.json file. Replace the values for the following keys with the IP address and port of your Helper LLM instance:

      • concept_perspective_generation
      • proposition_generation
      • concept_identification
      • filter_doc
      • dialog_summarization

  • 2.3 Start the text embedding model

    • Step 1: Install the required dependencies:

      Install cherrypy and sentence_transformers.

    • Step 2: Launch the embedding service:

      python text_embed_service.py --model gte_large --gpu YOUR_GPU_ID --port YOUR_PORT --batch_size 128

    • Step 3: Configure service URLs:

      Update the service_url_config.json file. Replace the value of sentence_encoding with the IP address and port of your text embedding service instance.

  1. Edit the configuaration files:

    docker-compose.yml:

    • add the "IP:Port" of the policy model (8b/70b) servers to SERVICE_IP;

Step 2: Start the system

  1. Download and install the Docker desktop/engine

  2. Go to the repo folder and then

docker-compose build

or

docker compose build
  1. Start the whole system with
docker-compose up

or

docker compose up

Then, you should be able to play with the system from your local machine at (http://0.0.0.0:8080). PS: for the first time you start the system, you might observe an error on some machines. This is because the database docker takes some time to initiate. Just kill it and restart, the error should be gone.

System Demonstration

Demo 1: Search for citations of an uploaded paper on Google Scholar.


Demo 2: Download a scientific paper and ask related questions.

Contribution

We welcome and appreciate any contributions and collaborations from the community.

Disclaimer

Cognitive Kernel is not a Tencent product and should only be used for research purpose. Users should use Cognitive kernel carefully and be responsible for any potential consequences.

Citation

If you find Cognitive Kernel is helpful, please consider cite the following technical report.

@article{zhang2024cognitive,
    title={Cognitive Kernel: An Open-source Agent System towards Generalist Autopilots},
    author={Zhang, Hongming and Pan, Xiaoman and Wang, Hongwei and Ma, Kaixin and Yu, Wenhao and Yu, Dong},
    year={2024},
}

Contact

For technical questions or suggestions, please use Github issues or Discussions.

For other issues, please contact us via cognitivekernel AT gmail.com.

Similar repos

CognitiveKernel/helper_1b_v1

Model

0

Tencent/KsanaLLM

549

same org

microsoft/JARVIS

JARVIS, a system to connect LLMs with ML community. Paper: https://arxiv.org/pdf/2303.17580.pdf

25,238

myshell-ai/AIlice

AIlice is a fully autonomous, general-purpose AI agent.

1,417

EmbraceAGI/LocalAGI

LocalAGI：Locally run AGI powered by LLaMA, ChatGLM and more. ｜ 基于 ChatGLM, LLaMA 大模型的本地运行的 AGI

81

AidenFaulconer/AI.DEN

Local agentic LLM using the latest and best in AI (MTP+Quant+Qwen3.5+caveman+claw-code)

2

superlinked/sie

Open-source inference server and production cluster for all the models your agent needs.

3,229

xingyaoww/code-act

Official Repo for ICML 2024 paper "Executable Code Actions Elicit Better LLM Agents" by Xingyao…

1,700

Languages

Python

66.6%

TypeScript

24.2%

JavaScript

8.6%