Building a Simple chatbot with Rust + Qdrant

Introduction

In this article, we'll explore how to use the Rust programming language and the vector database Qdrant to build a simple yet effective chatbot. The main point to cover a step by step guide that including setting up the development environment, implementing the chatbot logic, and integrating it with Qdrant for efficient data retrieval.

Why is Rust?

Rust is a popular programming language known for its performance, memory safety, and concurrency. It offers low-level control similar to C and C++ but with a focus on preventing memory errors through its ownership system. Rust's cross-platform support and robust package manager, Cargo, further enhance its versatility and ease of use.

Why is Qdrant?

Qdrant is a high-performance, cloud-native vector database and similarity search engine designed for AI applications. It is built using Rust, ensuring speed and reliability. Qdrant excels at handling high-dimensional vectors, making it ideal for tasks like semantic search, recommendation systems, and data analysis.

Technical Stack

• Backend: Rust with Axum web framework
• Vector Database: Qdrant
• AI Services: OpenAI API
• Authentication: Custom API key middleware
• Logging: Structured logging with tracing

Project Structure

src/
├── api/
│   └── routes.rs
├── handlers/
│   ├── chat.rs
│   └── embed.rs
├── middleware/
│   └── auth.rs
├── services/
│   ├── openai.rs
│   └── qdrant.rs
├── state.rs
├── types/
│   └── mod.rs
└── main.rs

Prerequisites

• Rust (latest stable version)
• Qdrant server
• OpenAI API key

Package Dependencies

Steps to Build the Chatbot

1. Initialize the Project

cargo new rust-qdrant
cd rust-qdrant

2. Add Dependencies Add the following to your Cargo.toml:

[package]
name = "rust-qdrant"
version = "0.1.0"
edition = "2021"

[dependencies]
axum = "0.7"
tokio = { version = "1.0", features = ["full"] }
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"
tower-http = { version = "0.5", features = ["trace"] }
tracing = "0.1"
tracing-subscriber = "0.3"
dotenv = "0.15"
async-trait = "0.1"
reqwest = { version = "0.11", features = ["json"] }
qdrant-client = "1.7"

3. Following code:

3.1 State Management (src/state.rs):

use std::sync::Arc;
use tokio::sync::RwLock;
use qdrant_client::client::QdrantClient;

pub struct AppState {
    pub qdrant: Arc<RwLock<QdrantClient>>,
    pub openai_api_key: String,
}

impl AppState {
    pub fn new(qdrant: QdrantClient, openai_api_key: String) -> Self {
        Self {
            qdrant: Arc::new(RwLock::new(qdrant)),
            openai_api_key,
        }
    }
}

3.2 Types (src/types/mod.rs):

use serde::{Deserialize, Serialize};

#[derive(Debug, Serialize, Deserialize)]
pub struct EmbedRequest {
    pub text: String,
}

#[derive(Debug, Serialize, Deserialize)]
pub struct ChatRequest {
    pub message: String,
}

#[derive(Debug, Serialize, Deserialize)]
pub struct ApiResponse<T> {
    pub data: T,
    pub error: Option<String>,
}

3.3 Authentication Middleware (src/middleware/auth.rs):

use axum::{
    http::{Request, StatusCode},
    middleware::Next,
    response::Response,
};
use std::env;

pub async fn auth_middleware<B>(
    req: Request<B>,
    next: Next<B>,
) -> Result<Response, StatusCode> {
    let api_key = req.headers()
        .get("x-api-key")
        .and_then(|v| v.to_str().ok())
        .ok_or(StatusCode::UNAUTHORIZED)?;

    let expected_key = env::var("API_KEY").map_err(|_| StatusCode::INTERNAL_SERVER_ERROR)?;
    
    if api_key != expected_key {
        return Err(StatusCode::UNAUTHORIZED);
    }

    Ok(next.run(req).await)
}

3.4 OpenAI Service (src/services/openai.rs):

use async_trait::async_trait;
use reqwest::Client;
use serde_json::json;

pub struct OpenAIService {
    client: Client,
    api_key: String,
}

impl OpenAIService {
    pub fn new(api_key: String) -> Self {
        Self {
            client: Client::new(),
            api_key,
        }
    }

    pub async fn create_embedding(&self, text: &str) -> Result<Vec<f32>, anyhow::Error> {
        let response = self.client
            .post("https://api.openai.com/v1/embeddings")
            .header("Authorization", format!("Bearer {}", self.api_key))
            .json(&json!({
                "model": "text-embedding-ada-002",
                "input": text
            }))
            .send()
            .await?;

        let embedding = response.json::<serde_json::Value>().await?;
        Ok(embedding["data"][0]["embedding"]
            .as_array()
            .unwrap()
            .iter()
            .map(|v| v.as_f64().unwrap() as f32)
            .collect())
    }
}

3.5 Qdrant Service (src/services/qdrant.rs):

use qdrant_client::client::QdrantClient;
use qdrant_client::qdrant::{PointStruct, SearchPoints};

pub struct QdrantService {
    client: QdrantClient,
}

impl QdrantService {
    pub fn new(client: QdrantClient) -> Self {
        Self { client }
    }

    pub async fn upsert_vector(
        &self,
        collection_name: &str,
        id: u64,
        vector: Vec<f32>,
    ) -> Result<(), anyhow::Error> {
        let point = PointStruct {
            id: Some(id.into()),
            vectors: Some(vector.into()),
            payload: None,
        };

        self.client
            .upsert_points(collection_name, vec![point], None)
            .await?;

        Ok(())
    }
}

3.6 API Routes (src/api/routes.rs):

use axum::{
    routing::{post, get},
    Router,
};
use crate::handlers::{chat, embed};
use crate::middleware::auth;

pub fn create_router() -> Router {
    Router::new()
        .route("/api/embed", post(embed::handle_embed))
        .route("/api/chat", post(chat::handle_chat))
        .layer(axum::middleware::from_fn(auth::auth_middleware))
}

3.7 Main Application (src/main.rs):

mod api;
mod handlers;
mod middleware;
mod services;
mod state;
mod types;

use axum::Router;
use dotenv::dotenv;
use std::env;
use tracing_subscriber::{layer::SubscriberExt, util::SubscriberInitExt};

#[tokio::main]
async fn main() {
    // Initialize logging
    tracing_subscriber::registry()
        .with(tracing_subscriber::EnvFilter::new(
            std::env::var("RUST_LOG").unwrap_or_else(|_| "info".into()),
        ))
        .with(tracing_subscriber::fmt::layer())
        .init();

    // Load environment variables
    dotenv().ok();

    // Initialize Qdrant client
    let qdrant_url = env::var("QDRANT_URL").expect("QDRANT_URL must be set");
    let qdrant_client = qdrant_client::client::QdrantClient::new(Some(qdrant_url));

    // Initialize state
    let state = state::AppState::new(
        qdrant_client,
        env::var("OPENAI_API_KEY").expect("OPENAI_API_KEY must be set"),
    );

    // Create router
    let app = api::routes::create_router()
        .with_state(state);

    // Start server
    let addr = "127.0.0.1:3000";
    tracing::info!("Starting server on {}", addr);
    axum::Server::bind(&addr.parse().unwrap())
        .serve(app.into_make_service())
        .await
        .unwrap();
}

4. Run and Test After implementing all components, you can run the application:

cargo run

The API will be available at http://127.0.0.1:3000 with the following endpoints:

• POST /api/embed - Generate embeddings
• POST /api/chat - Chat completions

5. Test Results and Comparison

• Initial State (Before Embedding)

# Test chat before any embeddings
curl -X POST http://127.0.0.1:3000/api/chat \
  -H "Content-Type: application/json" \
  -H "x-api-key: your_api_key_here" \
  -d '{"message": "Rust là ngôn ngữ lập trình gì?"}'

# Response
{
    "data": {
        "message": "I don't have specific information about Rust in my current knowledge base. Would you like me to search for information about Rust programming language?"
    },
    "error": null
}

• Document Embedding

# Store document with embedding
curl -X POST http://127.0.0.1:3000/api/embed \
  -H "Content-Type: application/json" \
  -H "x-api-key: your_api_key_here" \
  -d '{
    "text": "Rust là một ngôn ngữ lập trình hệ thống hiện đại, tập trung vào hiệu suất, an toàn và đồng thời. Nó ngăn chặn các lỗi segmentation và đảm bảo an toàn thread.",
    "metadata": {
        "title": "Giới thiệu về Rust",
        "category": "Programming",
        "language": "vi"
    }
}'

# Response
{
    "data": {
        "status": "success",
        "message": "Document embedded successfully",
        "metadata": {
            "title": "Giới thiệu về Rust",
            "category": "Programming",
            "language": "vi"
        }
    },
    "error": null
}

• After Embedding Test

# Test chat after embedding
curl -X POST http://127.0.0.1:3000/api/chat \
  -H "Content-Type: application/json" \
  -H "x-api-key: your_api_key_here" \
  -d '{"message": "Rust là ngôn ngữ lập trình gì?"}'

# Response
{
    "data": {
        "message": "Rust là một ngôn ngữ lập trình hệ thống hiện đại với các đặc điểm chính:\n\n1. Tập trung vào hiệu suất cao\n2. Đảm bảo an toàn trong lập trình\n3. Hỗ trợ lập trình đồng thời\n4. Ngăn chặn các lỗi segmentation\n5. Đảm bảo an toàn thread\n\nĐây là một ngôn ngữ được thiết kế để cung cấp hiệu suất tốt như C/C++ nhưng với các tính năng an toàn hơn."
    },
    "error": null
}

Conclusion

In summary, using Rust and Qdrant for building a chatbot offers a robust and scalable solution, though it may require some initial effort to master the technologies involved.

Good luck to your practice !!!

Pros

1. High Performance: Rust's speed and reliability make it ideal for high-load applications.
2. Scalability: Qdrant's cloud-native design allows for easy deployment and scaling.
3. Advanced Features: Qdrant offers advanced filtering, vector quantization, and distributed deployment, enhancing search efficiency and reducing memory usage.
4. Integration: Seamlessly integrates with various embeddings and frameworks, supporting multimodal data.
5. Open Source: Both Rust and Qdrant are open-source, providing flexibility and community support.

Cons

1. Learning Curve: Rust can be challenging to learn, especially for those new to the language.
2. Setup Complexity: Configuring Qdrant and integrating it with Rust requires careful setup and understanding of both technologies.
3. Resource Consumption: Running Qdrant in Docker may consume significant system resources, particularly for large-scale applications.

Feel free to expand on these points with more details and examples as needed. If you have any specific questions or need further assistance, let me know!

Reference Documents:

1. https://www.rust-lang.org/learn
2. https://qdrant.tech/documentation/
3. https://platform.openai.com/docs/api-reference/introduction
4. https://docs.rs/axum/latest/axum/
5. https://github.com/agapifa/rust-qdrant/