Scheduled Clinical Insights 🧬

AI-Assisted Quarto Reporting for Oncology Studies

Matthew Kumar

June 16, 2026

Matthew Kumar

Matthew Kumar

Associate Director, DSAI

Bayer

Home LinkedIn GitHub

Agenda

Problem Space

Previous Work

New Approaches

Demo

Future

Problem Space

Background

  • Early oncology studies bring significant complexity, from novel therapies to unconventional trial designs and vulnerable patient populations
  • Timely insight from emerging data is essential for high-impact development decisions
  • Traditional reporting tools are often too constrained for exploratory or fast-changing analytical needs
  • Bespoke analysis requests can create delays that are misaligned with decision timelines
  • The need is for a faster, more adaptable reporting approach that still maintains rigor in a regulated environment

Previous Work

What we did

  • Started with a deterministic Shiny review app for baseline and efficacy exploration
  • Added an experimental LLM layer based on querychat for natural-language filtering and data analysis
  • Developed ellmer tools to create novel figures and tables (with R code returned) and one-click interpretation of standard plotsellmerggplot2Tplyr
  • A complementary approach worked well: a Shiny foundation for reliable review, with the LLM accelerating exploration and driving insight in the last mile
  • PHUSE US Connect 2026 paper

What we learned

  • Users generally liked the LLM features, especially plot interpretation as a quick “second set of eyes”
  • Reception was mixed: some users were less comfortable with open-ended AI generation
  • For some reviewers, a full Shiny app was more than they needed; they already knew the exact views they wanted

Thinking…

  • A simpler path: a static scheduled report with fixed visuals, plus LLM insights, delivered to the reviewer inbox

New Approaches

Data Process

Quarto on Connect

Scheduled Quarto doc lives and runs on Posit Connect

Query Data

Queries the SQL warehouse for the most recent oncology trial data

Save as Pin

Saves results to Posit Connect as a versioned Pin dataset

Report data ready

Primary use feeds a second scheduled Quarto report on Connect

Secondary uses include ad hoc analysis and additional reporting

Report Process

Read Pin

Quarto report reads the pinned dataset from Connect

Build visuals

Quarto renders oncology charts, graphs, and tables from the pin data

LLM interprets

ellmer sends chart images to the internal LLM service for narrative interpretation

Render report

Interpretation returned, woven into the Quarto document and stored on Connect. Source published, previous reports accessible

Email to users

Report e-mailed automatically to users on schedule

LLM Process

Study documents

Ragnar processes study documents (e.g. CSP, SAP, IDRP) and others into context

Saves context in a reusable vector store (DuckDB)

LLM interprets

ellmer sends chart images to the internal LLM service for narrative interpretation; study information provided as context

Render report

Richer interpretations returned, grounded in study context

Demo

Click here to view report

Human In The Loop (HIL)?

Future

Directions

  • Most of this experiment is for feasibility

  • Continue the journey by refining the pipeline, prompts, and guardrails as we learn

    • vitals is a framework for evaluating LLM outputs with ellmer
    • Systematic prompt comparisons, model grading, and inspectable eval logs

  • Explore pins storage backends, including AWS S3 buckets

  • Embed prior reports in the RAG process to surface the delta and journey of issues, findings, and insights over time

  • Onboard new users to gauge receptiveness and gather further feedback

Thank you