RocketMQ-Rust

English | 简体中文

🚀 A high-performance, reliable, and feature-rich unofficial Rust implementation of Apache RocketMQ, designed to bring enterprise-grade message middleware to the Rust ecosystem.

---

✨ Overview

RocketMQ-Rust reimplements Apache RocketMQ in Rust: the NameServer, Broker, Controller and Proxy services, an async client SDK, and the tooling to operate them. It keeps RocketMQ's model of topics, queues, consumer groups and NameServer routing, speaks the RocketMQ remoting protocol, and serves the gRPC MessagingService through its Proxy, while relying on Rust's ownership model for memory safety and on Tokio for asynchronous I/O.

Status: 1.0.0 is the current stable release. All 28 workspace crates are published on crates.io, and 13 container images are published on Docker Hub and GHCR. RocketMQ-Rust is an independent community distribution, not an official Apache Software Foundation release.

🎯 Why RocketMQ-Rust?

• 🦀 Memory Safety: Rust's ownership model removes whole classes of bugs such as use-after-free and data races at compile time, with no garbage-collection pauses
• ⚡ Async by Design: Services and clients run on Tokio with explicit runtime ownership, bounded queues and admission control, and graceful shutdown that reports what it stopped
• 🔁 RocketMQ Compatible: Implements the remoting wire protocol and the gRPC MessagingService, and converts existing Java Broker .properties files; Apache RocketMQ 5.5.0 is the comparison baseline
• 💾 Pluggable Storage: Local CommitLog and ConsumeQueue files by default, with optional RocksDB and tiered storage backends
• 🔒 Secure and Observable: ACL authentication and authorization, TLS transport, and OpenTelemetry metrics, traces and logs with OTLP and Prometheus exporters
• 🛠️ Operations Included: Admin CLI and terminal UI, web and desktop dashboards, container images, Helm charts and Kubernetes manifests
• 🤖 AI-Ready Operations: A read-only MCP server for diagnostics, a separately controlled MCP server for supervised changes, and an AI SRE platform
• 🌐 Cross-Platform: Built and tested in CI on Linux, Windows and macOS

🏗️ Architecture

RocketMQ-Rust is built from the components below. A minimal cluster needs only a NameServer, a Broker and a client; the rest is optional.

• NameServer: Broker registration, liveness tracking and topic route lookup
• Broker: Request processing, topic and consumer-group metadata, message storage and delivery
• Store: The storage engine inside the Broker, with CommitLog, ConsumeQueue and index files plus optional RocksDB and tiered backends
• Client SDK: Producers and push, lite pull and POP consumers for Rust applications
• Proxy: gRPC and optional remoting ingress, in front of a cluster or with an embedded Broker
• Controller: OpenRaft-based master election and replica coordination for controller-mode high availability
• Operations tooling: Admin CLI and TUI, dashboards and MCP servers, all off the message path

The architecture overview covers the message lifecycle, storage and high-availability design.

📚 Documentation

• 📖 Official Documentation: rocketmqrust.com - Getting started, architecture, configuration, deployment and operations guides, in English and Chinese
• 📝 API Docs: docs.rs/rocketmq-client-rust - API reference for the client SDK
• 📋 Examples: rocketmq-example and rocketmq-client/examples - Ready-to-run producer and consumer samples
• 🧭 Design Documents: rocketmq-doc - Architecture decision records, protocol notes and migration guides
• 📦 Releases: GitHub Releases and CHANGELOG.md - Release notes and change history
• 🤖 AI-Powered Docs: DeepWiki - Interactive documentation with intelligent search

🚀 Quick Start

Run a single-node cluster from source and exchange messages. Every listener binds to 127.0.0.1, and the Broker keeps its data in the git-ignored .rocketmq/ directory.

Prerequisites

• Git and a Rust toolchain installed through rustup; the repository pins Rust 1.95.0 in rust-toolchain.toml
• A C/C++ build toolchain for native dependencies: MSVC Build Tools on Windows, Xcode Command Line Tools on macOS, or GCC/Clang on Linux
• Free TCP ports 9876, 10909, 10911 and 10912, and about 1.5 GB of disk space for the Broker's preallocated store files

git clone https://github.com/mxsm/rocketmq-rust.git
cd rocketmq-rust

Run every command below from the repository root, with each service in its own terminal. The first build takes several minutes.

1. Start the NameServer

cargo run -p rocketmq-namesrv --bin rocketmq-namesrv-rust -- --rocketmqHome .rocketmq --bindAddress 127.0.0.1

The NameServer listens on 127.0.0.1:9876. It needs a home directory: pass --rocketmqHome or set ROCKETMQ_HOME, otherwise it exits during startup.

2. Start the Broker

Create the .rocketmq directory and save this configuration as .rocketmq/broker.toml:

[broker]
namesrvAddr = "127.0.0.1:9876"
brokerIp1 = "127.0.0.1"
storePathRootDir = "./.rocketmq/broker"

[broker.brokerServerConfig]
bindAddress = "127.0.0.1"

[store]
storePathRootDir = "./.rocketmq/store"
haListenAddress = "127.0.0.1"

cargo run -p rocketmq-broker --bin rocketmq-broker-rust -- -c .rocketmq/broker.toml

Wait for the started successfully log line. The Broker registers with the NameServer and listens on 127.0.0.1:10911. Without a configuration file it binds to all interfaces, advertises an auto-detected address and stores its data under ~/store.

3. Send and Receive Messages

cargo run -p rocketmq-client-rust --example simple-producer

The producer sends ten messages to TopicTest, which the Broker creates on first use, and exits. Then start the consumer:

cargo run -p rocketmq-client-rust --example consumer

The consumer prints the ten messages and keeps waiting for more. Press Ctrl+C to stop it.

4. Stop and Clean Up

Press Ctrl+C in the Broker terminal, let its shutdown finish, then stop the NameServer the same way. Delete .rocketmq/ to discard the data.

This walkthrough runs without authentication or TLS, which is why every listener stays on loopback. Read the deployment security guide before exposing a service to a network.

Use the Client SDK in Your Application

Add the client and the three crates it is used with:

[dependencies]
rocketmq-client-rust = "1.0.0"
rocketmq-model = "1.0.0"
rocketmq-runtime = "1.0.0"
rocketmq-observability = "1.0.0"

The application owns the runtime and hands it to every client. A minimal producer:

use rocketmq_client_rust::{ClientRuntime, ClientRuntimeConfig, DefaultMQProducer};
use rocketmq_model::common::message::message_single::Message;
use rocketmq_observability::TelemetryRuntimeGuard;
use rocketmq_runtime::RuntimeOwner;

fn main() -> Result<(), Box<dyn std::error::Error>> {
    // The application owns the runtime; the client never creates a hidden one.
    let owner = RuntimeOwner::new()?;
    let telemetry = TelemetryRuntimeGuard::noop();
    let client_runtime = ClientRuntime::try_new(
        owner.root_context().component("example-client"),
        ClientRuntimeConfig::default(),
        telemetry.handle(),
    )?;

    let (sent, client_report) = owner.block_on(async {
        let mut producer = DefaultMQProducer::builder(client_runtime.clone())
            .producer_group("example_producer_group")
            .name_server_addr("127.0.0.1:9876")
            .build();
        let sent = async {
            producer.start().await?;
            let message = Message::builder()
                .topic("TopicTest")
                .body_slice(b"Hello RocketMQ")
                .build()?;
            producer.send_with_timeout(message, 3000).await
        }
        .await;
        // Shut down in order: producer, shared client runtime, then the owner.
        producer.shutdown().await;
        (sent, client_runtime.shutdown().await)
    });
    let runtime_report = owner.shutdown_runtime_blocking()?;

    println!("send result: {:?}", sent?);
    if !client_report.is_healthy() || !runtime_report.is_healthy() {
        return Err("client shutdown did not complete cleanly".into());
    }
    Ok(())
}

Consumers follow the same ownership pattern. Continue with:

• Client guide - Push and lite pull consumer examples, Cargo features and TLS
• First-message tutorial - The same path with an explicitly created topic and consumer group
• rocketmq-example - Ordered, delayed, transactional, batch and request/reply messaging
• Rust API migration guide - Source changes when upgrading from 0.x

📦 Components & Crates

The root Cargo workspace contains 28 crates, all published on crates.io at the workspace version. Directory and package names match, except that the rocketmq-client directory holds the rocketmq-client-rust package. Dashboards, AI operations products, examples and the website are standalone projects with their own manifests.

Core Runtime Services

Client, Protocol, and Shared Libraries

Storage, Runtime, and Observability

Operations Tooling

Standalone Projects

These projects live in this repository but outside the root workspace. Build each one from its own directory.

💡 Capabilities

Availability depends on Cargo features, runtime configuration and the selected storage backend or topology. The capability matrix lists the conditions for each capability.

🚢 Deployment

Release images use the same component names in both registries:

docker.io/mxsm/rocketmq-rust-<component>:<version>
ghcr.io/mxsm/rocketmq-rust/<component>:<version>

The components are namesrv, broker, controller, proxy, mcp, dashboard-web-backend, dashboard-web-frontend, sre-control-plane, sre-connector, sre-executor, sre-execution-agent, sre-probe and sre-ui. Images ship without deployment configuration: a core service reads /etc/rocketmq/<component>.toml and keeps its data under /var/lib/rocketmq, so mount both. See the release and image guide for details.

🧪 Build & Validation

Cargo selects packages by name, so use rocketmq-client-rust for the rocketmq-client directory.

A full workspace build also compiles RocksDB and the Proxy's protobuf bindings, so it additionally needs Clang (libclang) and protoc, either on PATH or through the PROTOC environment variable. Standalone projects under rocketmq-example/, rocketmq-dashboard/, rocketmq-ai/, rocketmq-website/ and fuzz/ are built and validated from their own directories.

🤝 Contributing

We welcome contributions from the community! Whether you're fixing bugs, adding features, improving documentation, or sharing ideas, your input is valuable.

How to Contribute

1. Pick an open issue or create one from the issue templates; issues labeled good first issue are scoped for newcomers
2. Fork the repository and create a branch from main
3. Change one thing at a time, add or update tests, and run the checks for the crates you touched
4. Record user-visible changes under Unreleased in CHANGELOG.md
5. Open a Pull Request against main that links the issue, with a title such as [ISSUE #1234]📝Clarify consumer offset completion

Contribution Guidelines

• Follow Rust best practices and the engineering rules in AGENTS.md, which apply to people and AI coding agents alike
• Add tests for new functionality
• Update documentation as needed
• Keep pull requests focused and leave unrelated code untouched
• Follow the Code of Conduct

For detailed guidelines, please read CONTRIBUTING.md and our Contribution Guide.

Repository Activity

❓ FAQ

Is RocketMQ-Rust production-ready?

1.0.0 is the first stable release. The services ship with health probes, graceful shutdown, security profiles and observability, but production readiness also depends on your workload, topology and failure model. Validate the message paths you rely on and work through the production checklist before going live.

Is it compatible with Apache RocketMQ?

RocketMQ-Rust implements the RocketMQ remoting protocol and, through the Proxy, the gRPC MessagingService, with Apache RocketMQ 5.5.0 as the comparison baseline. Compatibility is defined per surface, not as a blanket guarantee: the Controller runs its own OpenRaft protocol and cannot join a Java Controller quorum, DLedger is not supported, and a Java Broker's data directory is not a drop-in store. See the protocol and compatibility reference.

What's the minimum supported Rust version (MSRV)?

Rust 1.95.0. Every workspace manifest declares it as rust-version, and rust-toolchain.toml pins the same stable version for builds.

How does performance compare to Java RocketMQ?

No head-to-head figures are published, because throughput depends on hardware, storage, message size, and flush and replication policy. The Criterion benchmarks in the crates (cargo bench -p <package>) track regressions in individual components. For sizing, measure your own workload and read capacity and performance.

Can I use it with existing RocketMQ deployments?

Yes, at the client boundary. A Rust application can use rocketmq-client-rust against existing NameServers and Brokers over the remoting protocol, and gRPC clients can connect through the Proxy. Verify the operations you depend on against your exact server version, ACL and TLS settings. Rust and Java services do not share Controller quorums or data directories.

How can I migrate from Java RocketMQ to RocketMQ-Rust?

Choose the migration boundary first, because each one is a different operation:

1. Move applications to the Rust client SDK while keeping the existing cluster
2. Run a separate Rust cluster and shift traffic to it step by step
3. Replay or bridge historical data explicitly instead of reusing a Java data directory

The Broker can load an existing Java broker.properties file and report how each setting was converted. Refer to the migration guide for detailed steps.

👥 Community & Support

• 💬 Discussions: GitHub Discussions - Ask questions and share ideas
• 🐛 Issues: GitHub Issues - Report bugs or request features
• 📧 Contact: Reach out to mxsm@apache.org

Contributors

Thanks to all our contributors! 🙏

📄 License

RocketMQ-Rust is licensed under the Apache License 2.0.

See LICENSE-APACHE and NOTICE, or http://www.apache.org/licenses/LICENSE-2.0.

🙏 Acknowledgments

• Apache RocketMQ Community for the original Java implementation and design
• Rust Community for excellent tooling and libraries
• All Contributors who have helped make this project better

---

_{Built with ❤️ by the RocketMQ-Rust community}