Incident sheet · /enterprise-ai
- System
- Enterprise AI
- Filed
- Read
- 4 min · 849 words
- Severity
- Routine
ICU mortality model scores 0.864 in U.S., drops to 0.698 in China
ELDER-ICU hit 0.864 AUROC in U.S. and Austrian ICUs but 0.698 in China. Recalibration helped every site; only retraining recovered the biggest ranking losses.

Event log
- ELDER-ICU scored 0.804–0.864 AUROC in U.S./Austrian ICUs but 0.753 in South Korea and 0.698 in China.
- Retraining on local data recovered +0.066 AUROC in South Korea and +0.076 in China.
- The study ran at 12 ICU sites across four countries, published in npj Digital Medicine.
- Findings were cited at Telehealth Connect Summit 2026 in Tunis, Sept. 29–Oct. 1.
- Recalibration via isotonic regression and Platt scaling improved calibration at every site.
One ICU mortality prediction model scored 0.804 to 0.864 AUROC in U.S. and Austrian hospitals — and 0.698 at a site in China. That 0.166-point spread, reported in npj Digital Medicine for the machine learning tool ELDER-ICU, turns a research finding into a purchasing problem for every hospital signing a contract for AI trained on another health system's patients.
The numbers surfaced in Telehealth.org's Nov. 9 report by Julia Ivanova, which covered the Telehealth Connect Summit 2026 held in Tunis from Sept. 29 to Oct. 1. The report drew on attendee Maya Sherne, Telehealth.org's general manager. At the summit, health ministries, the World Health Organization, the International Telecommunication Union, the Global Digital Health Partnership and private technology companies gathered with one shared goal: getting digital health and AI out of pilot mode.
What does the study actually show?
ELDER-ICU predicts in-hospital death for ICU patients aged 65 and older, using clinical variables recorded in the first 24 hours after admission. Its developers originally built and validated it on data from the United States and Amsterdam. In this round, researchers tested it at 12 ICU sites across the U.S., Austria, South Korea and China, checking both geographic differences and shifts over time, including the COVID-19 period.
The results split sharply:
- U.S. and Austrian cohorts: AUROC between 0.804 and 0.864
- South Korean site: 0.753
- Chinese site: 0.698
- Site mortality rates ranged from roughly 6% in South Korea to about 22% in China, with all other sites between 10% and 21%
AUROC measures how well a model ranks patients by risk — 0.5 is a coin flip, 1.0 is perfect separation of patients who died from those who survived. Calibration, a separate test of whether stated probabilities match real outcomes, matters when clinicians act on a specific risk percentage.
The authors' conclusion: rigorous validation and proactive adaptation are essential before clinical use wherever populations and clinical practice vary.
Which fix matches which failure?
The study gets most useful after the drop. Researchers tried three update strategies, and each asks more of the hospital than the last:
- Recalibration — the lightest option. Isotonic regression and Platt scaling adjust output probabilities to match local outcomes without rebuilding anything. Both improved calibration at every site.
- Incremental training — fine-tuning the model on newly collected local data while keeping its core architecture. It improved performance at most centers, and the authors note the method has seen little use in critical care so far.
- Retraining — the heaviest lift, and it delivered the biggest gains where they were needed most: AUROC rose by 0.066 at the South Korean site and 0.076 at the Chinese site, the two places the original model had slipped furthest.
The practical order of work follows directly. Measure performance on local patients first. Recalibrate when rankings hold but stated probabilities are off. Save retraining for sites where the ranking itself breaks down. Each step needs something different: outcome data the hospital can label, people to run the update, and contract terms that allow the model to be changed at all.
Why does local validation so often go undone?
The Multilateral Perspectives on Digital Health panel at the Tunis summit offered part of the answer: agreeing on technical standards doesn't mean anyone applies them. Speakers described fragmented projects piling up when health ministries, telecom agencies and private companies each build systems on their own. Their proposed remedy — national interoperability standards, digital public infrastructure and government oversight.
The panel also framed AI adoption in three moves: adopt an existing model, adapt it to the national context, then advance it with local expertise and data. The real point was capacity. Countries — and, by extension, individual hospitals — need people who can evaluate AI systems themselves rather than relying entirely on technology built somewhere else.
In the U.S., that capacity gap shows up in rural planning. A May 2026 National Consortium of Telehealth Resource Centers webinar on state Rural Health Transformation Fund plans noted that rural providers increasingly rely on technical and operational support that may not be fully reflected in current workforce strategies. For AI specifically: if no one on staff can run a local validation, a hospital is left relying on whatever the vendor tested.
The summit's final day turned to execution, with workshops on building interoperable health systems and designing telehealth and AI projects that move from planning to implementation. A teleradiology session with participants from Egypt, Tunisia and the WHO focused on cross-country exchange — panelists argued countries can learn from each other rather than copying models built in wealthier systems.
Before the next AI contract goes out for signature, the ELDER-ICU numbers suggest three questions: ask for performance results from sites whose patients look like yours, whether the license allows local recalibration or retraining, and who runs and pays for that work after go-live.
At the Chinese site, retraining bought back 0.076 AUROC — but only a license and an assigned owner can make that work possible.
via marketscale.com (Source)
More from Olivia Hart
Show full bio
News editor covering money, funding and market shifts around enterprise information technology and the leaders who run at CIO Monitor.
207 articles
The newsroom digest
One email a day with the stories that matter. No noise.
Cross-references
- Bangkok hospital and Australian tiler show how AI returns real revenue
- Apple Bets on a Study, Not a Product: 200,000 Users to Test Clinical AI
- Healthcare AI governance lags behind rising patient-data risk
- Singapore Care Agency Breach Exposes Records of 4,064 Workers
- Medical device maker breach notifications reach nearly 4 million