With predictions for global healthcare shortages making headlines, it's no surprise that many in the industry focus on finding ways to reduce the clinical burden. AI and machine learning are front and centre of many discussions, raising questions about how to safely and effectively implement these technologies. The industry's regulated nature means these conversations are often imbued with a certain level of anxiety: How can we adapt quickly enough? How do we ensure we take adequate precautions as we move forward? What data strategies have helped other healthcare companies handle the risk?
The fundamental guidelines for healthcare data governance and AI continue to evolve. In an article for Data Science Magazine, we lay out the critical standards that healthcare needs to adhere to in moving forward with these technologies, namely:
- The EU-AI Act
- IEC's General Technology Standards
- ISO/IEC 23894: Guidance on AI Risk Management
- ISO/IEC 42001:2023 Guidance for Management Systems
- FDA Artificial Intelligence/Machine Learning Guidance for SaMD
Below, we will showcase a case study that preceded many of these guidelines because technology's evolution often outpaces regulations. We must adopt frameworks as an industry to help us push innovative solutions during critical periods, particularly when it means saving lives and reducing the clinical burden.
Governing bodies must know that companies are building safe systems that mitigate risk. Adopting some of the existing best practices that uphold industry-wide principles of quality, safety, and effectiveness makes it possible to design solutions with AI and machine learning that can benefit society. The case study below shows that remaining risk-averse and pragmatic is critical.
Case study | Building a clinical support tool to provide urgent care during COVID-19
COVID-19 was remarkable for many reasons, one of which was the pressure it put on the global healthcare system to evolve rapidly in response to the crisis. We're all familiar with the story: there wasn't time for the slow development of solutions. Hospitals scrambled to provide urgent care. At an unprecedented pace, organizations managed to produce vaccines and clinical support tools to save lives.
At S3 Connected Health, we worked closely with a team of senior respiratory consultants to create Enodatis — a clinical support tool to help hospitals optimize and scale acute respiratory care. It supported clinicians (respiratory specialists and non-specialists) using machine learning to mine patient data and quickly provide accurate predictions about the likelihood of needing vital ICU care or equipment such as a ventilator. When the healthcare system was overwhelmed, it allowed for optimizing and forecasting resource requirements in providing care (including staff and equipment).
At the time, there were fewer guidelines around deploying machine learning in a medical device or platform. Much of what now exists was only present in more general guidance covering all industries. To roll out Enodatis promptly, we relied on rigorous risk assessment in accordance with any existing guidelines around machine learning (making pragmatic assumptions as to how these might translate when brought over to the healthcare industry).
The team developed recommendations and security centered around patient safety. Using existing AI guidelines from other fields, it was possible to imagine what they might look like for sensitive healthcare data.
Despite predating the official requirements, Enodatis meets the HTI-1 guidelines, showing that even when the exact guidelines for a new technology don't exist, experience and expertise can adequately assess risk and ensure that new solutions are fair, appropriate, valid, effective, and safe.
Four critical steps we took to ensure success:
- Followed a risk-based approach using standard risk management principles for creating medical devices.
- Defined the scope and mathematical model that approximated COVID-19 so we could build a system concentrated on lung function. The model demonstrated how the disease operated, and we built our expectations of graduation through treatment on that.
- Used clinical panels to review what we were doing at every stage of development and for post-market launch.
- Expanded the clinical panel after it was rolled out so we could frequently update it based on regular feedback.
As the development of Enodatis during the COVID-19 pandemic shows, the guidelines aren’t always ready, but adhering to best practices can demonstrate a quality-based approach.
There will be times when delivering urgent care through medical devices takes precedence or situations where our healthcare systems need urgent innovations to prevent the system from becoming overwhelmed. If you’d like to find out more about the existing regulations and best practices, our recent Data Science Magazine article, The Role of Data Science, Machine Learning, and AI in Reducing the Clinical Burden, provides a full outline.
