Enormous volume of documents

The first challenge in this project was taking over an already initiated development. Due to a shortage of resources on the client side, the responsibility for completing and optimizing LiLi was placed in our hands. This initially required an intensive deep dive into the existing software. We had to thoroughly analyze the existing code structure and identify numerous technical hurdles. It quickly became evident that the original development state was not optimally designed for the planned scaling, necessitating a fundamental reassessment of many components and extensive refactoring.

At the same time, the massive volume of documents passing through the system daily posed another challenge. The incoming data originated from various sources and appeared in sometimes unpredictable formats — ranging from standardized PDFs and scanned documents to unstructured content from emails or very large pictures from mobile phones. The continuous growth in document flow significantly increased the system’s performance demands.

With each new source integrated, the load on the system would increase further. When larger document inflows were activated, it became clear just how enormous the data volumes hitting the system were. It quickly became evident that without a clear strategy to tackle this challenge, bottlenecks and delays would be inevitable.

LiLi needed to be not only flexible and powerful but also capable of handling this diversity and volume — at maximum speed.

High-performance, high-speed application

The implementation was carried out in close collaboration with the clients project manager and his team. The original, extensive Business Requirement Document (BRD) was transformed into an agile, practical structure: a focused Kanban board with clearly prioritized tasks ensured that work was always concentrated on the most relevant to-dos. This allowed us to respond quickly to new requirements, identify bottlenecks, and continuously implement optimizations without losing track of progress.

It was particularly important to find a solution that addressed both the challenges of the existing software structure and the exponentially growing volume of documents. The initial codebase required extensive refactoring to optimize performance and eliminate bottlenecks. At the same time, the system needed to scale with increasing workload without compromising speed or stability.

To efficiently process the enormous number of documents, it was essential to develop a powerful architecture capable of seamlessly integrating various input sources. The key was to convert diverse documents — ranging from scanned letters and emails to social media messages — into a uniform, machine-readable format. This was achieved through a combination of specialized services working in parallel, each handling a clearly defined subprocess. This distributed processing approach ensured that LiLi remained performant at all times and could process large volumes of data at high speed.