Build NLP for Production
Evaluate any Transformer model. Combine Haystack components into flexible pipelines. Build services from common NLP tasks. Deploy as NLP backend API.
123456789101112131415161718# Build a simple Question Answering pipelinep = ExtractiveQAPipeline( retriever=dpr_retriever, reader=roberta_reader)p.run( query="What is the outlook for the US market?") # Returned answer:{'answers': [{'answers': '+22% growth in revenue', 'context': "For the US market, we expect a strong Q4 with +22% growth in revenue.", 'probability': 0.989, ....},....Latest Models
Pick any Transformer model from Hugging Face's Model Hub, experiment, find the one that works.
Flexible Document Store
Use Haystack NLP components on top of Elasticsearch, OpenSearch, or plain SQL.
Vector Databases
Boost search performance with Pinecone, Milvus, FAISS, or Weaviate vector databases, and dense passage retrieval.
Scalable
Build semantic search and question answering applications that can scale to millions of documents.
End-to-end
Building blocks for the entire product development cycle such as file converters, indexing functions, models, labeling tools, domain adaptation modules, and REST API.
Pipelines
It's not one-size-fits-all! Combine nodes into flexible and scalable pipelines and launch powerful natural language processing systems.
When to add NLP to your applications
From an idea to a natural language search — address the varying demands of business users with ease. Below are some examples of use case-driven applications for Haystack.
Ready to get started?
Start learning Haystack
Frequently asked questions
Haystack is a flexible open source Python NLP framework to build powerful, scalable NLP backend services. It enables the developers to apply Transformer models to real-world technical use cases — semantic search, open-domain question answering, information extraction, summarization, answer generation, conversational AI and more. Haystack is being developed by deepset.
The Transformer is a deep learning model initially open sourced by Google in late 2018. It sparked a wave of interest from researchers and developers alike because of its groundbreaking performance. Thanks to a community of researchers and NLP practitioners, many new variants of the Transformer model appeared.
Our community consists of NLP researchers, experts, but also of NLP practitioners, and even full-stack developers who build semantic search and question answering with Haystack. We welcome everyone who's interested in natural language processing and is solving a real-world use case involving natural language processing.
To quote Wikipedia: "Question answering (QA) is a computer science discipline within the fields of information retrieval and natural language processing (NLP), which is concerned with building systems that automatically answer questions posed by humans in a natural language."
Haystack is designed to be a very practical, down-to-earth open source NLP framework. A decent knowledge of Python would be a prerequisite, and some fundamentals of machine learning too. We work on lowering the barrier of entry to NLP, and we always appreciate sharing your experiences with us. For inference, you'd also need a GPU-enabled system.
Haystack framework is completely open source, licensed under Apache License 2.0. As deepset-the-company, we've already helped the largest European companies and public sector organizations to instrument semantic search, and we're also building an enterprise SaaS product to complement Haystack. We'd be very happy to discuss this in more detail.