Bursar

From the Discworld series written by Sir Terry Pratchett

A Streamlit application for parsing and analyzing transportation contracts using OpenAI.

Setup

1. Install Poetry if not already installed.

2. Run poetry install to install the necessary dependencies

3. Create a .env file in the root of the project and add your API keys. Here is an example:

   OPENAI_API_KEY="your_openai_api_key"
   GEMINI_API_KEY="your_gemini_api_key"
   

How to Run

To run the app, execute the following command from the root of the project:

streamlit run app.py

Challenges

Analyzing and parsing transportation contracts presents several challenges, primarily stemming from two key issues:

1. Inherent Complexity of Contracts: The contracts themselves are often complex documents that require deep domain knowledge of the client's specific operations to be interpreted and set up correctly within our system.
2. Unique Data Modeling: Our internal system utilizes a specific and unique data model for representing slab-wise information, which makes direct extraction and mapping from a raw contract document a non-trivial task.

To mitigate the first challenge, this project currently focuses on parsing only the simplest types of contracts seen in our system.

Evolution of the Approach

Our initial approach was to use a powerful language model to directly extract information from the contract documents and format it according to our internal entity structure. This internal model is defined in resources/contracts.json and resources/contracts.yaml (sourced from Locus's Swagger documentation). We attempted to provide this schema to the model by storing it in a vector database and making it accessible via a file search tool during the extraction process.

However, this method proved to be ineffective. The models struggled to perform the implicit calculations and reasoning necessary to bridge the gap between the contract's text and our structured format. Even advanced reasoning models failed to produce accurate results consistently.

This led to a revised, two-step approach:

1. Extraction to an Intermediate Structure: The first step now focuses solely on extracting the slab information from the contract into a simpler, intermediate format. It's noteworthy that reasoning-focused models perform significantly better at this task than other models.

2. Computation and Transformation: Once the slab information is extracted, a custom-written method processes this intermediate structure to compute and format the data to fit into the Locus entity layer, specifically for the slab steps.

This decoupled approach has yielded slightly consistent results, demonstrating that a combination of model-based extraction and programmatic transformation is more effective for this complex task.