I'm building another one, a new app. This time I'll share my progress and findings with you. I'm building a Lithuanian law assistant with LLM using RAG. As nowadays usual, it is a chat interface which must be aware of multiple Lithuanian law entities and relationships between them to answer questions: how is this law act's paragraph applied in practice? What are the consequences? What is the process? I genuinely believe that this application will be useful and the application of LLM + RAG is a good fit here (let me know if you think the opposite).

## Tech Stack

- PostgreSQL + Prisma

- Node.js + NestJS

- Vite, React (Radix UI components)

- Vitest, pnpm, TypeScript

- Models: text-embedding-3-small, gpt-4o-mini

I am using NestJS for the first time. First impression: this is a Java Spring framework implemented in TypeScript. It has a rich ecosystem, and for now it is a good fit for a small project.

## What I've Done So Far

I have a chat interface in dark theme with calming green elements. I haven't encountered any complexity with the UI at all. It is simple and it has one screen.

I've learned the ingestion pipeline meaning. Very important part of the application. One of the most important. What is ingestion? This is a process of transforming any data to structured data with embeddings (vectors) for later retrieval. The quality and accuracy of responses depends on the quality of data. For example, you want to ingest labor law. Then you need to find one, export it as any type of artifact (docx, pdf, html), parse it to text (I've chosen markdown), and then chunk it to entities (act → article → paragraph), create embeddings for those chunks and save embeddings to the database.

### Parsing and Chunking

Currently parsing and chunking are the most time-consuming tasks for me. Current implementation can ingest any Lithuanian act (act → article → paragraph). This step requires thorough testing and multiple iterations. At this point regular expressions are my solution for parsing and chunking.

### Conversation

For conversation I'm doing the following:

- Combining history to one prompt - the tradeoff: multiple LLM calls and longer response time, but more accurate context augmentation

- Get embeddings for this prompt

- Query my chunks by embedding - I store only paragraph chunks, because article chunks may be too long, so I've omitted embedding article as a whole. Though I store reference to an article

- Query LLM with chat history (last 12 messages) + chunks from DB - not sure that 12 last messages is a good solution here. But I need to make a decision how to prevent context overflow

For now this process works as expected for chat interface. Response time is acceptable, but database and servers are deployed locally. Deployed application latency may add some milliseconds. At this point it is fast enough.

## What Are My Next Steps?

I will work on expanding the domain, adding legal documents, procedures, institutions, decisions, concepts, principles and consequences. This is not the hardest "business" domain. But for me there is a lot what I should learn first.

Adding new entities to the application requires:

- Understanding why it is important

- Understanding what this entity does

- Parsing and chunking data

- Storing and linking to existing domain entities

Building a Lithuanian Law Assistant with LLM and RAG (part 1)

### **What is RAG?**

RAG stands for Retrieval Augmented Generation. This is the process when the Large Language Models (LMM) input is enriched with a contextual information from the external sources by a provided prompt.

### **What problem it solves?**

- Stale or outdated data. Retrieving relevant data from the external sources and data which is up to date.

- Invalid answers or hallucinations. Adds data which may be missing, new or proprietary.

### **What is the Process?**

The RAG process consists of three main stages: _Retrieval_

- orchestrator accepts the prompt in a free text form. _"Where are libraries in my district?"_

- creates a vector query from the prompt. Vector is also called vector embedding. Vector is a floating point number (1.345). Vector embedding is a series of vectors (e.g. **`[1.34, 4.78, -0.35]`**).

- vector query is used to retrieve information from vector database.

- vector database can match similar vector by a vector query and returns top matching documents or text passages _Augmented_

- The retrieved text passages are integrated into the original prompt, adding specific context. For example: _"Context: Libraries in users district are located in X street 1km away form his location. Question: Where are libraries in my district?"_ _Generation_

- LLM generates response based on prompt and context

### **Is Vector database mandatory?**

No, a vector database is not mandatory for the RAG process. However, it is highly recommended due to its efficiency, scalability, and high accuracy in matching semantically similar texts, significantly improving the quality and reliability of retrieval results.