How I Connected Live Monitoring, a Lakehouse, Early-Warning Scoring and a Simulation Study in One AWS Project

The Data and the System Goal

I built my healthcare realtime project to connect three responsibilities: show incoming measurements, retain them for analysis and score encounters with an approved model. The paths share event contracts and timing rules while serving different purposes. A live measurement, an analytical row and a model score need different handling even when they begin with the same event.

The project runs on Amazon Web Services (AWS). It uses fictional patient records generated with Synthea and public waveform-derived heart rate, breathing rate and oxygen saturation from Beth Israel Deaconess Medical Center (BIDMC). Every 5 minutes the simulator adds blood pressure and three bedside measures the public recordings lack: temperature, inhaled oxygen and consciousness. Those three are synthetic. AWS runs use a cohort of 10 patients; local development uses 100 from the same Synthea run, with the AWS cohort as its first 10. The model label is a synthetic deterioration proxy. Each deteriorating encounter gets its own values inside fixed ranges; normal encounters vary inside the normal ranges. The project architecture defines the system as an engineering demonstration. It provides no basis for patient-care decisions.

That scope guides the account. I can show how an observation passes through the system, how its history supports analysis, how model controls govern scoring and how a simulation study tests the models. Those are engineering claims. They do not establish clinical validity.

This account describes the project as released. The release record documents what was accepted on AWS, from an end-to-end session through a load test to teardown. The changelog records each change. The simulation study runs on a local stack instead of AWS.

Following an Event Through Live Monitoring

Consider a hypothetical heart-rate observation from the simulator. This example illustrates the implemented path; it is not a measurement collected during a new run for this article.

The simulator represents the measurement as a Fast Healthcare Interoperability Resources (FHIR) Observation. In this project, that resource carries the measurement and its relationship to a patient and encounter. HAPI FHIR receives it, then a subscription invokes the webhook. The webhook checks its shared secret and publishes a normalized event to Amazon Kinesis Data Streams. Kinesis supplies the stream consumed by the realtime processor and the analytical delivery path. The architecture shows those connections.

I keep identity and timing in the event because the downstream services need more than a numeric value. The observation identifier supports duplicate handling. Patient and encounter identifiers connect the event to the right context. The event timestamp determines whether a measurement should update the latest state. The star-schema guide documents the observation grain and the patient and encounter keys used in analytical joins.

The processor is an AWS Lambda function that updates the patient’s latest measurements in DynamoDB. Its implementation uses observation claims for idempotency, meaning that repeated delivery should not be treated as a new observation merely because it arrives again. The processor reserves an observation, marks a successful claim complete and releases it after a retryable error. An expired reservation can be reclaimed. A claim token stops one worker from completing another worker’s reservation. That control has a defined lifetime and failure behavior; it is not a promise of exactly-once delivery across the whole system.

Latest-state updates also need to handle partial observations. A heart-rate event should not erase an oxygen-saturation value simply because that value is absent from the incoming payload. The processor updates the vital fields present in the event and records their timestamps. Conditional updates check the incoming time against the stored time for those fields, which protects the merged snapshot from an older update to the same vital. The snapshot represents the latest accepted information for each vital, not a claim that every measurement was taken simultaneously.

The processor keeps a second record as well. Each encounter’s readings go into a feature-window table, including readings the latest-state snapshot skips because a newer value already arrived. The model needs the whole first 15 minutes of an encounter, not only the newest value, so the two tables answer different questions. Feature-window rows expire after two days, long enough to score any encounter the simulator produces.

The processor sends accepted updates to clients through a WebSocket application programming interface (API), a connection that lets the server push messages. A separate request-based API reads the latest snapshot.

The same API has an early-warning endpoint. It returns the National Early Warning Score 2 (NEWS2), a standard bedside score built from seven measurements, computed from the patient’s current values. After an encounter’s 15-minute window closes and a 60-second grace period allows for late readings, it also returns the approved model’s score for that window. The scoring runs in plain Python from the parameters training publishes. A test checks it matches the training library’s own score. The endpoint stores one score per encounter and model version, so asking again returns the same answer instead of a fresh calculation.

The Streamlit monitoring interface keeps the simulated cohort visible and distinguishes current from stale measurements. It shows each patient’s live NEWS2 with its risk band, from the measurements still current, alongside the approved model’s encounter probability when available. It also counts the patients scoring 5 or above. I treat freshness as part of the displayed information: a plausible value needs its timing context before a reader can interpret it. Live NEWS2 and the early-warning endpoint each passed a check on AWS, separate from the release checks.

Building Analytics From the Same Stream

The latest-state snapshot answers a narrow question: what accepted measurements are available for this patient? It does not replace the history needed for analysis. Amazon Data Firehose delivers stream events into Amazon Simple Storage Service (S3). Those stored events preserve the normalized identifiers and measurements; they are not full copies of the original FHIR resources. The analytical architecture documents that distinction.

AWS Glue prepares the stored events for querying. It classifies measurements, separates rejected rows into a quarantine table and merges accepted observations into an Apache Iceberg table. Amazon Athena queries the registered data. I want to be able to explain why an event failed a preparation rule before considering whether it should re-enter processing, so quarantine and controlled correction stay separate steps.

The analytical transformations use dbt to build an observation fact table and related dimensions, including patient, encounter and provider context. A fact table holds measurements at a defined grain; the dimensions provide the descriptive context used to filter and group them. Each simulated admission has a facility, a unit, an admitting diagnosis and a fictional attending provider. A reference extract in the daily workflow writes that context to S3 as JSON Lines: demographics, payer history, facilities, admissions and the groups of patients that share a waveform recording. Glue tables read it, dbt joins it on AWS and Soda contracts check each reference model. One of them counts each unit’s census, admissions and discharges by hour. The star schema guide defines the grain and join paths.

Two more models prepare the comparison the study needs. One scores NEWS2 at the end of each encounter’s feature window from the last value of each measurement; it stays empty when any measurement is missing. The other records each vital’s trend as a slope over the window’s minute averages. A check compares both against an independent Python calculation from the raw records.

Retaining an event does not make it eligible for every downstream dataset. The staging path limits analysis to the configured cohort and observations with valid encounter identity, while raw history stays available for audit and replay. This separates preservation from analytical selection.

The same dbt models also run on a local PostgreSQL warehouse, loaded from batch files the simulator’s own code generates. That lets me build and test the study’s 100-patient data without an AWS deployment, as the changelog describes. Local results do not prove AWS behavior, so I keep the two kinds of evidence apart.

Quality checks run within the workflow. Athena validation and Great Expectations run before dbt, then controlled scoring, prediction refresh and Soda checks follow. A key relationship, a processed-data condition and the freshness of a published table need different assertions. Apache Airflow and Amazon Managed Workflows for Apache Airflow Serverless coordinate that sequence. OpenLineage events connect the executed stages through a shared run identity, so I can ask which run produced the dataset I am inspecting.

Adding Controlled Scoring and Reporting

The model path begins with eligibility and timing. Its dataset contract takes features from the first 15 minutes of an encounter and builds the synthetic proxy label from the following 15. Encounters that do not support both windows are excluded. Separating the windows keeps the feature summary apart from its later outcome.

The proxy label reflects repeated extreme readings of the same vital. It is a simulation target defined by the project, not a verified clinical outcome. I use logistic regression as a baseline for that target. The training dataset in Structured Query Language (SQL) assigns partitions deterministically, so a patient’s encounters stay on one side of the split. Training requires both label classes in both partitions before it proceeds.

Training and approval are separate from routine scoring. The daily workflow consumes an explicitly approved model version; it does not retrain automatically. The scorer checks the published artifact’s checksum, manifest version, feature order and scoring contract before building predictions. A checksum answers whether the artifact matches the recorded bytes. The manifest answers which model version and inputs those bytes belong to. Neither shows that the label represents a clinically meaningful outcome.

The reporting paths stay separate. The live dashboard shows measurements, NEWS2 and the approved model’s encounter probability when available. The model analytics dashboard ranks the cohort by proxy probability and shows each score with its encounter and feature-window context. It also shows the approved model, the decision threshold, the label definition, freshness and the clinical-validation status. It cannot change the live dashboard’s warning priorities.

Grafana adds three operational dashboards, Pipeline Operations, Quality and Safety and Capacity, the last with an hourly census of each unit. I wrote each dashboard once and render it for either warehouse, so the same panels read PostgreSQL locally and Athena on AWS. The renderer rejects SQL that only PostgreSQL accepts, which keeps the shared source honest. A local check runs every panel query through Grafana and compares the census panel with the warehouse. Grafana runs on the local stack. On AWS it is optional and stays private behind a port forward; it passed a check there too. The Power BI connection guide keeps the Power BI report private, so it is not public evidence of a completed dashboard.

Testing the Models Against a Known Answer

A model trained on synthetic data raises an obvious question: what would a good result even mean? I answered it by building the answer in. The simulation study plants a seeded early-warning trend in the first 15 minutes of most deteriorating encounters, with a smaller heart-rate drift for patients aged 65 and over. A null control arm plants nothing. If a model finds signal in the null control, something other than the planted trend is leaking into it. The planted trend gives the study a known answer: I know which encounters carry it and how strong it is, so I can judge whether a method recovers it.

Before the study itself, I calibrated the signal’s strength on a separate seed, without the study’s own data. On that seed, a logistic regression reached an area under the curve (AUC) of 0.802, where 0.5 is chance and 1.0 is perfect separation. NEWS2 at the end of the window reached 0.691. On the null control the two reached 0.501 and 0.480, close to chance as they should be. The calibration also confirmed that the smaller heart-rate drift for older patients is detectable: across ten extra seeds, its 95% confidence interval stayed below zero every time. The changelog records the calibration.

The analysis itself compares NEWS2 with three models under a frozen plan: a naive logistic regression, a logistic model that accounts for repeated measurements of the same patient and gradient-boosted trees. The run checks the plan’s hash before reading any data. Only encounters with all seven NEWS2 measurements enter the comparison, so NEWS2 is never scored on partial data. Every model is scored out of fold, grouped so that two patients who share a waveform recording never sit on both sides of a split. Intervals come from a bootstrap that resamples those whole groups. A standard test for correlated curves compares each model with NEWS2 on the same encounters. The plan records each deviation and when it was made, including one made after study results were seen.

The most useful finding came before any model ran. The warehouse’s new NEWS2 cross-check exposed a problem in the planted trend: it had been pushing waveform dropouts, an oxygen saturation of zero or a breathing rate below 4, up to the lowest valid value. The processor would have rejected those 537 readings. They appeared only in deteriorating encounters, so they marked the label. A model could have learned the leak instead of the signal. The changelog records the fix: dropouts stay as they are, so the processor rejects them.

The study’s answers describe how well a method recovers a planted signal in synthetic data, not clinical performance.

Operating and Verifying the Whole Project

I treat the failure paths as part of the system’s design, because invalid records and retryable failures need different handling. The architecture documents permanent rejection, an encrypted failure queue, bounded replay and a separate dead-letter queue for events that cannot be recovered through that path. Replay is a controlled recovery operation, not a claim that every failure resolves automatically.

A second workflow checks ingestion every 30 minutes when its schedule is on. It confirms that the webhook answers, that HAPI’s subscription to it is active and that the webhook had no errors. A failed check raises an alarm. A silent ingestion failure would otherwise look like a quiet cohort. The workflow passed a check on AWS.

The same architecture separates realtime observability from analytical pipeline monitoring. A successful analytical task does not establish that clients are receiving live updates. A changing dashboard does not establish that the historical models passed their checks. After a failure, the recovery model calls for inspecting the underlying issue and validating state, dashboard behavior and alarms. An empty queue alone is not recovery.

The release notes record the AWS acceptance. With an approved model, one workflow run completed all ten tasks. An isolated load test lost no observation; WebSocket delivery took 241 ms at the median and 1,381 ms at the 95th percentile. The model trained for that acceptance had four test encounters. That exercises the deployment path but cannot support a predictive-performance claim; the simulation study is where model comparison belongs.

Each deployment is a short-lived demonstration, so I keep its infrastructure state on the deploying machine. Teardown removes that state after the managed deployment is destroyed and Terraform state lists no resources. Account prerequisites, such as the Grafana admin secret, remain. The repository’s infrastructure lifecycle guide covers reproducible infrastructure and controlled teardown. Demonstrating the project includes knowing how it starts, how failures are investigated and how disposable resources are removed.

Three lessons guide how I explain the build:

Those lessons explain the connected system better than a service list. The components matter because they implement the contracts and boundaries that let monitoring, analysis, scoring and the study work together.

See the case study: Healthcare Realtime Monitoring