How to Choose a News API

With hundreds of thousands of news articles being published daily, businesses and researchers use news data in their analysis to draw valuable insights. This is enabled by the availability of news data in bulk, and by technical and AI advances such as natural language processing (NLP) and LLMs.

In any news analysis workflow, a news data vendor plays a key role. Firstly they have the infrastructure to gather huge volumes of news articles from various sources. Second, they also enrich the news data by cleaning it up, analyzing sentiments, clustering them, and more. Picking the right news data vendor can be crucial to your success in your news analysis endeavor. In this blog post, we discuss how to select an appropriate news data vendor for your use case.

Common Use Cases for News Data

Risk Analysis

Businesses use news data to flag risks that could affect their operation or cause market disruptions. Analyzing global and regional news can identify risks such as global health crises and cybersecurity threats that every business needs to be prepared for. In addition to these, there are narrow-impact, industry-specific risks such as changes in the prices of certain raw materials or changes in economic policies that could affect only your business. Companies involved in finance, VC, and equity also monitor the news for information related to stocks and holdings.

Market Research

News data vendors provide clean, sorted news articles that can be filtered by topics of relevance, and this could save valuable work hours while doing market research. They also pre-analyze and enrich the data in practical ways—by deduplicating, grouping, analyzing sentiments, and more. They can also provide historical news data dating back years or even decades, which can be difficult to get from the source. This is helpful in analyzing trends. News data monitoring also keeps you informed about the current market situation and your competitor’s actions so you can be prepared for any potential disruptions.

AI & LLM

LLM APIs are getting cheaper with each passing quarter, but the most popular models are trained on datasets that could be a few years old. With the latest news data, you can enrich your LLMs to be up to date, and they can give accurate responses with much broader contexts.

Media Monitoring

Businesses and government agencies constantly monitor the news to analyze how their public perceives their brand and actions. This can provide good, actionable feedback. Good News APIs can help with this by providing sentiment analysis, comprehensively collecting articles from various sources, and grouping articles by different viewpoints.

Useful Features in a News Data Workflow

Data Quality and Coverage

In terms of news data itself, a good vendor must be able to scrape as much data as legally and technically possible from a huge repository of reliable sources.

• Full Text: Does the vendor provide the full text of news articles?
• Comprehensive Coverage: Given a news event, can they provide as many articles related to it as possible from the widest range of sources?
• Source Reliability: Are the news sources reliable? Is there a source reliability ranking or filter?

Data Access

Once the data is scraped and processed by the vendor, the next thing to consider is how you can access the news data that you need.

• Filtering: Does the vendor provide an option for filtering by keywords, publication date, language, country, source rank, and so on? The more the filtering capabilities, the better.
• Low Latency: How long does it take between a news article being published and the vendor scraping it and making it available?
• Historical Data: How many years of historical news data can they provide?
• Data Delivery: What data delivery options are available? Some options are: pulling via APIs, receiving periodic data dumps to cloud storage, and receiving streamed news data with low latency.

Enrichments

Apart from raw news data, vendors can also use NLP and AI to pre-analyse and enrich the news data. This can save time and effort on your end.

• Clustering & Deduplication: Can the vendor cluster related news articles and deduplicate articles talking about the same news event?
• Tagging: Are the articles tagged with a topic, e.g.: Cybersecurity Threats? Is it possible to filter the articles using this tag?
• Resolving Keyword Clashes: When a keyword has multiple meanings, can the vendor distinguish news articles accordingly and provide only relevant articles? e.g.: Amazon (rainforest) vs Amazon (company)
• Sentiment Analysis: Are they analyzing news articles to determine the sentiment? Do they provide news articles with a sentiment tag such as positive, negative, accepting, critical, and so on?

Working with the Vendor

Having a good experience working with the vendor is as important as getting good and enriched news data.

• UX: Is it easy to perform trivial tasks such as managing access tokens, checking usage, and making payments?
• Flexibility & Scalability: Can the vendor accommodate your specific needs such as monitoring particular sources? Can the vendor continue to support you if your scale of operation changes?
• Support: Is the documentation of the vendor platform good? Does the vendor have satisfactory support channels with good turnaround times?

For elaborate descriptions of each feature and how it can add value to your news data analysis workflow, read our white paper.

Picking a Vendor

Things to Consider

After shortlisting vendors based on the features they offer, you may want to evaluate them based on operational and pricing factors.

• Service Stability: It’s best to pick vendors who provide SLAs to ensure that the API is always up and running. This will prevent disruptions in your news workflow due to vendor server outages.
• Flexibility: You might want the vendor to monitor additional sources or deliver data in a specific format. Can the vendor accommodate that? Also, does the vendor provide any client libraries that make integration easy for your engineers?
• Support: Despite good documentation and great UI, there can always be a need to contact support to resolve certain issues. It’s good to pick vendors who are easy to contact via email, chat, etc.
• Pricing: Consider what features are being offered at the price point the vendor is quoting. It’s best to evaluate this from the perspective of value added for the price paid. Having clean data with enrichments might be worth the extra bucks if it saves time on your end.

Pointers for a Demo Call

• Is the vendor able to suggest appropriate features and an integration plan based on your use case?
• Who are the other customers of this vendor? Do they have expertise in your industry and can they share the best practices? Do they have recent case studies that are relevant to your industry?
• Do they have all the crucial features for your news analysis workflow? For example, if you are interested in media monitoring, do they provide low latency access, sentiment analysis, and wide coverage (per source and number of sources)?
• Does the vendor give you trial credits or allow for a pilot project? Can they show relevant sample data?
• What is the pricing and what are the features they provide at that price point?
• How will you integrate the API into your workflow? Do they have good documentation and any client libraries (Python, Java, etc.) that you could use? Do they provide support for integration?
• Does the vendor provide SLAs for guaranteed uptime?

Conclusion

In this blog, we briefly discussed how to go about selecting a news data vendor based on your use case. Working with a good vendor can help you extract the best value from the thousands of news articles being published each day. We at NewsCatcher are committed to supporting your news analysis journey. To learn more about how you can leverage news data by choosing the right set of features and picking the right vendor, read our white paper. If you are ready to take the first step or are looking to switch from your current vendor, please contact us to schedule a demo call.