Parisian AI startup Pathway, led by a female team, has just lifted the curtain on its innovative data processing engine. Boasting an impressive speed up to 90 times faster than current streaming solutions, Pathway’s engine sets a new record as the fastest data processing mechanism on the market.
Pathway’s data processing marvel
Their secret? An unmatched capability to combine batch and streaming logic within the same workflow, allowing the system to discard obsolete data, akin to human memory mechanisms.
Pathway’s CEO and co-founder, Zuzanna Stamirowska, highlights the historical separation of batch and streaming architectures due to their inherent complexities. This division, she says, hinders the advancement of data streaming in AI systems, cementing their intelligence in a static state. The presence of a third workflow, generative AI, further complicates matters.
The data processing revolution and the rise of adaptable AI
Stamirowska emphasizes the urgent need for rapid data processing and a more versatile AI. Pathway is focused on facilitating real-time data processing while ensuring a seamless experience for developers, regardless of whether they handle batch, streaming, or LLM systems.
An essential feature of the new engine is its ability to “forget” incorrect or outdated information in real-time, which previously was a daunting challenge due to models being trained on static data. Pathway’s solution allows data point revisions without necessitating a complete batch data upload.
Making waves in the tech industry
Pathway, founded in 2020 by the Polish-French team of Stamirowska and Claire Nouet, the company’s COO, has already made significant strides. After a successful pre-seed funding round in December, it has expanded to over 20 employees across Europe and North America. With this latest innovation, Pathway aims to establish a universal language for all data pipelines, democratizing streaming workflows for developers and reducing costs for clients.
We’d love to hear your thoughts on this exciting development in the field of data processing. What impact do you think Pathway’s innovative engine will have on the tech industry? Share your insights and opinions in the comments below!
