by LitWatch (LX AI)
LitWatch digests only the papers that match your question. Decision-support — not a systematic review.
I start by pulling the latest feeds from arXiv and PubMed every hour. The raw XML is parsed into a common record format that keeps the title, abstract, authors, categories, and a link to the full text. I keep only the fields needed for matching to reduce memory use and speed up the scan.
Each record is run through a lightweight text matcher that looks for exact phrase hits and stemmed keyword hits from your query. I use a simple Boolean engine: terms must appear in the title or abstract, and you can require that certain words appear together. The matcher returns a score based on term frequency and position, which I then compare to a threshold you set.
When a record passes the threshold, I push it to your chosen channel: email digest, Slack webhook, or RSS feed. The digest groups matches by day and includes a short summary line with the title, authors, and a direct link. You can also export the day’s matches as a CSV for offline review.
I recommend writing your question as Population, Intervention, Comparison, Outcome (if applicable). For example, “In adults with type 2 diabetes, does a low‑carb diet compared to a standard diet improve HbA1c after six months?” This structure gives you concrete terms to include in the query.
From the PICO statement, extract synonyms for each concept. For the population you might use ["type 2 diabetes", "T2DM", "adult diabetes"]. For the intervention you might use ["low carbohydrate diet", "low‑carb", "carb restricted"]. Keep each group as a list of alternatives separated by OR inside the matcher.
If you want to limit to certain article types, add tags like “[Title]” or “[MeSH]” (for PubMed) or “[cat:]” (for arXiv). I expose these tags in the query builder so you can restrict matches to titles only, which cuts noise when you are looking for method papers.
Store the final query string in your LitWatch dashboard. After a week of runs, review the false positives and adjust synonyms or thresholds. The system logs each match decision so you can see why a paper was kept or dropped.
Precision is the fraction of returned papers that are truly relevant. Recall is the fraction of all relevant papers that you capture. Raising the threshold improves precision but can drop recall. Lowering the threshold does the opposite. I provide a precision‑recall curve after each run so you can see the effect of moving the cutoff.
You can give extra weight to terms that appear in the title. For example, a title hit counts as two points while an abstract hit counts as one. This biases the matcher toward papers that explicitly discuss your concept up front, which often raises precision without a large recall loss.
Add a NOT list for concepts you never want to see, such as “review”, “meta‑analysis”, or “editorial”. The matcher subtracts points for any NOT term found, pushing those records below the threshold. This is a quick way to cut down on noise from generic survey articles.
| Setting | Effect on Precision | Effect on Recall |
|---|---|---|
| High threshold (≥0.8) | Increases | Decreases |
| Title weight ×2 | Increases slightly | Small decrease |
| Add NOT “review” | Increases | Decreases (removes survey) |
| Low threshold (≥0.4) | Decreases | Increases |
I built a “dry run” mode that returns the top 20 scored papers without sending them out. You can view the list, check the scores, and adjust the threshold in real time. Once you are satisfied, switch back to live mode and the digest will start delivering.
LitWatch is not a replacement for a full systematic review. It does not guarantee that you will see every relevant paper, nor does it claim to eliminate all bias. The tool is meant to support decision‑making by surfacing items that match your stated criteria, but you still need to read the full text and judge applicability yourself. It does not provide statistical synthesis, risk‑of‑bias tables, or meta‑analytic results. In short, it is a focused alert service, not a comprehensive evidence‑review pipeline.
I set the email digest to arrive at 08:00 local time. I skim the list, flag any papers that look pertinent, and move the links to a reading list in my reference manager. If a paper needs deep review, I add a note with the question it addresses and schedule a 30‑minute block later in the day.
The Slack webhook posts a short message with the title and link to a dedicated channel. Team members can react with an emoji to signal interest, and I use those reactions to prioritize which papers to discuss in our weekly journal club.
Every time I edit the query, LitWatch saves a new version with a timestamp. I can roll back to a previous version if a change introduces too many false positives. The history view shows the number of matches per day for each version, making it easy to spot the impact of a tweak.
At the end of each month I download the CSV of all matches and store it in our project folder. This gives us an audit trail of what surfaced and lets us verify that we did not miss a major update due to a mis‑configured filter.
When you connect external services like Slack or email, LitWatch uses OAuth tokens that are stored encrypted on disk. I rotate these tokens every 90 days and log any failed authentication attempts. For details on securing LLM‑based components, see the OWASP Top 10 for LLM Applications https://owasp.org/www-project-top-10-for-large-language-model-applications. If you are evaluating the overall risk of adding an automated alert system to your research pipeline, the NIST AI Risk Management Framework offers useful guidance https://www.nist.gov/itl/ai-risk-management-framework. I follow the framework’s advice to document data flows, assess impact, and set up periodic reviews.
I keep a written checklist next to every release that touches this topic. The list is boring on purpose: name the owner, name the evidence artifact, and name the rollback. If a step cannot produce a file, a ticket, or a log line, it is not a step — it is a wish.
| Signal | What to record | Stop-ship? |
|---|---|---|
| Unexpected write or send | Tool name, args, identity | Yes until gated |
| Missing confirmation on destructive action | Call path and bypass | Yes |
| Untrusted text treated as instruction | Source surface and resulting action | Yes for agent reach |
| Scanner clean, judgment open | Deferred items + owner | Only if core path safe |
litwatch helps with the measurable first pass. Architecture choices, legal sign-off, and customer-facing claims still need a human owner. Treat every checklist row as decision support, not a certificate.
When a prospect asks whether the product "covers everything," I answer with the boundary: automated checks catch known patterns; novel chains and policy judgment still need review. That honesty keeps support load down and keeps the blog aligned with what the product actually does.
If you need a second opinion, pair this write-up with the references below and with your own staging reproduction. Do not skip the reproduction step just because a model wrote a confident summary.
LitWatch digests only the papers that match your question. Decision-support — not a systematic review.
Want to try the product? LitWatch
Generated 2026-09-17 · primary sources only · no fabricated traffic metrics