Skip to content

on_boarding.md

Latest commit

 

History

History
280 lines (235 loc) · 49.8 KB

on_boarding.md

File metadata and controls

280 lines (235 loc) · 49.8 KB
graph LR
    Core_AI_Services["Core AI Services"]
    Data_Management["Data Management"]
    Document_Processing_System["Document Processing System"]
    Chat_System["Chat System"]
    Evaluation_System["Evaluation System"]
    System_Utilities_Observability["System Utilities & Observability"]
    CLI_Orchestration["CLI & Orchestration"]
    Guardrail_Service["Guardrail Service"]
    Document_Processing_System -- "utilizes" --> Core_AI_Services
    Document_Processing_System -- "enriches via" --> Core_AI_Services
    Document_Processing_System -- "accesses" --> Data_Management
    Document_Processing_System -- "fetches data from" --> Data_Management
    Chat_System -- "communicates with" --> Core_AI_Services
    Chat_System -- "compresses via" --> Core_AI_Services
    Evaluation_System -- "uses" --> Core_AI_Services
    Evaluation_System -- "generates via" --> Core_AI_Services
    Evaluation_System -- "produces data for" --> Document_Processing_System
    CLI_Orchestration -- "invokes" --> Document_Processing_System
    CLI_Orchestration -- "manages" --> Data_Management
    CLI_Orchestration -- "interacts with" --> Core_AI_Services
    CLI_Orchestration -- "executes via" --> Core_AI_Services
    CLI_Orchestration -- "initiates" --> Evaluation_System
    Guardrail_Service -- "moderates via" --> Core_AI_Services
    Data_Management -- "depends on" --> Core_AI_Services
    System_Utilities_Observability -- "supports" --> Core_AI_Services
    System_Utilities_Observability -- "monitors" --> Core_AI_Services
    System_Utilities_Observability -- "supports" --> Data_Management
    System_Utilities_Observability -- "monitors" --> Data_Management
    System_Utilities_Observability -- "supports" --> Document_Processing_System
    System_Utilities_Observability -- "monitors" --> Document_Processing_System
    System_Utilities_Observability -- "supports" --> Chat_System
    System_Utilities_Observability -- "monitors" --> Chat_System
    System_Utilities_Observability -- "supports" --> Evaluation_System
    System_Utilities_Observability -- "monitors" --> Evaluation_System
    System_Utilities_Observability -- "supports" --> CLI_Orchestration
    System_Utilities_Observability -- "monitors" --> CLI_Orchestration
    System_Utilities_Observability -- "supports" --> Guardrail_Service
    System_Utilities_Observability -- "monitors" --> Guardrail_Service
    click Core_AI_Services href "https://github.com/CodeBoarding/GeneratedOnBoardings/blob/main/ragbits/Core AI Services.md" "Details"
    click Data_Management href "https://github.com/CodeBoarding/GeneratedOnBoardings/blob/main/ragbits/Data Management.md" "Details"
    click Document_Processing_System href "https://github.com/CodeBoarding/GeneratedOnBoardings/blob/main/ragbits/Document Processing System.md" "Details"
    click Chat_System href "https://github.com/CodeBoarding/GeneratedOnBoardings/blob/main/ragbits/Chat System.md" "Details"
    click Evaluation_System href "https://github.com/CodeBoarding/GeneratedOnBoardings/blob/main/ragbits/Evaluation System.md" "Details"
    click System_Utilities_Observability href "https://github.com/CodeBoarding/GeneratedOnBoardings/blob/main/ragbits/System Utilities & Observability.md" "Details"
    click CLI_Orchestration href "https://github.com/CodeBoarding/GeneratedOnBoardings/blob/main/ragbits/CLI & Orchestration.md" "Details"
    click Guardrail_Service href "https://github.com/CodeBoarding/GeneratedOnBoardings/blob/main/ragbits/Guardrail Service.md" "Details"
Loading

CodeBoardingDemoContact

Component Details

The ragbits project provides a comprehensive framework for building and evaluating Retrieval-Augmented Generation (RAG) systems. It integrates various AI services like LLMs and embeddings, manages data storage and retrieval, and offers robust document processing capabilities. The system includes a chat interface, an extensive evaluation framework, and a command-line interface for interaction, all supported by core utilities and observability features, with an optional guardrail service for content moderation.

Core AI Services

Provides fundamental interfaces for interacting with Large Language Models (LLMs) for text generation, streaming, and token counting, and manages the creation of dense and sparse vector embeddings from diverse data types. It also handles the construction, rendering, and parsing of prompts for LLMs.

Related Classes/Methods:

Data Management

Offers a standardized interface for storing, querying, and managing vector data across different vector database implementations and provides mechanisms to connect to and retrieve data from various external and local sources.

Related Classes/Methods:

Document Processing System

The primary component responsible for orchestrating document search operations, managing the end-to-end process of ingesting documents (parsing, enriching, indexing), and improving search results by rephrasing user queries and reranking retrieved documents. It also defines the structured representation of documents and their constituent elements.

Related Classes/Methods:

Chat System

Provides the public API and internal interface for managing chat conversations, including handling user inputs, generating responses, and managing chat state. It also handles the storage and retrieval of chat conversation history.

Related Classes/Methods:

Evaluation System

The overarching framework for evaluating RAG systems, encompassing the execution of evaluation pipelines, metric computation, result logging, and optimization of evaluation parameters. It also provides tools for generating synthetic datasets and loading evaluation datasets.

Related Classes/Methods:

System Utilities & Observability

Provides foundational utility functions for configuration management, data transformations, and dependency validation. It also implements tracing and metric collection functionalities to monitor the execution flow and performance of the system.

Related Classes/Methods:

CLI & Orchestration

The main entry point for the ragbits command-line interface, handling command parsing, execution, and output formatting. It also manages the execution of agents, which are high-level workflows or decision-making units within the RAGbits system.

Related Classes/Methods:

Guardrail Service

Provides safety and moderation functionalities, such as integrating with external moderation APIs.

Related Classes/Methods:

FAQ