Navigating the Challenges of Building an AI AutoScribe with a DOM EHR Integration


At Beam Health, we’ve always been obsessed with delivering solutions that streamline healthcare operations. When we decided to pivot into AI-based auto-scribing with the introduction of Shine AI, we knew we were diving headfirst into uncharted territory. Shine AI was designed to be more than just a utility. We envisioned it as a game-changer that could lift the cumbersome burden of clinical documentation off healthcare providers. This is not a feat for the faint-hearted and involves overcoming technical obstacles at multiple levels—from the front-end challenges of DOM manipulation to the backend intricacies of formatting complex prompts that not only generate detailed clinical notes, but also play nice with the EHR’s formatting requirements.

The Front-End: Navigating the Unpredictable Waters of DOM Manipulation

If you’re not a coder, DOM (Document Object Model) manipulation might sound like arcane wizardry. But in simpler terms, think of inserting clinical notes into an EHR system like adding a paragraph to a complex, ever-changing book. Each EHR has its own “chapters” or sections where notes can go, and these sections often contain other elements like checkboxes or drop-down menus.

Sounds daunting? Now imagine doing it in multiple books with different styles and structures. That’s what we’re dealing with when trying to insert clinical notes into various Electronic Health Records (EHR) systems through the DOM.

Dealing with Asynchronicity

Things on the web don’t happen one after the other in a neat line; they often happen all at once or in an order we can’t predict. This creates a problem when we try to post the note to the EHR. We had to ensure that while the new note is being added, no other existing elements are altered or deleted.

Staying Consistent with Different EHRs

Different EHRs have different rules for formatting and presenting clinical notes. Some want them in a bulleted list; others want them in a structured table. It’s like adding a paragraph to a book, but each book has a different publisher with a different style guide. We had to build a system intelligent enough to recognize these various styles and adapt accordingly.

The Balancing Act

Putting all these pieces together required an almost orchestral level of coordination. Every component had to be harmonious with the others, while also being flexible enough to adapt to new “instruments,” or in our case, new EHRs. The challenge was in finding the right technology stack that gave us this balance between flexibility and control.

The Backend: Wrestling with EHR Formatting and Language Models

We chose Langchain as our framework to generate complex prompts and formulate our language models. The framework comes with tremendous potential, enabling us to fine-tune our models to suit the specific formatting requirements of different EHRs. Sounds like a dream, right? Not so fast.

Langchain, being a relatively new framework, brings with it a steep learning curve. Unlike more established frameworks, it doesn’t have an ocean of community support or documentation to dip into when you’re stuck. Every challenge becomes a learning opportunity, and every hurdle involves a lot of trial and error.

Don’t get me wrong – I’m all for emerging technologies that push the envelope. However, the absence of extensive documentation and examples meant that we had to explore a lot of uncharted terrain ourselves. That said, the flexibility it offers in generating language models that align with EHR-specific formatting is unparalleled, and the struggle was worth the end result.

Wrapping It Up

Navigating the technical maze to build Shine AI was not a walk in the park. It required a synergy of innovative thinking, technical proficiency, and an unwavering commitment to the healthcare industry.

I couldn’t be prouder of what we have achieved. As we continue our journey, I invite you to stay tuned. The challenges (and opportunities!) are far from over, and we are just getting started.