Challenges of Data in Healthcare

In today’s fast-paced and data-driven world, many businesses leverage the prowess of data analytics to improve performance, increase profits, and identify their shortcomings. 

Like every other sector, healthcare can also reap these benefits, especially in big hospitals and patient record keeping. However, the biggest problem in this regard is the fact that challenges faced in data analytics and management in the healthcare industry don’t only lead to financial loss but also the loss of human life. 

Therefore, it’s important to be fully familiar with the underlying complications and challenges of big data analytics before utilizing it in a healthcare facility or space. 

According to an International Data Corporation study, the healthcare industry is likely to see the fastest growth rate among all other sectors, such as manufacturing and entertainment, included in the report. 

However, here’s the catch; the healthcare industry received a 2.4 DATACON score, which is a standard for measuring the data growth, management, security, and readiness in any industry. At 2.4, the healthcare sector falls below average in terms of data competency, technological innovation, and IT investment. 

But this is not the only challenge in the healthcare industry. Below, we discuss some of the challenges the healthcare sector faces in big data analytics and management. 

Data Collection 

Nowadays, healthcare institutions are using electronic health records to manage information and capture data from patient appointments. Additionally, the data includes personal health records, reimbursement information, patient portals, and much more. 

As one may anticipate, collecting this tremendous amount of data is a difficult task. The data doesn’t only need to be collected but should also be in an easy-to-scan-through format so that healthcare providers can use it to make decisions. 

Here are some challenges the healthcare industry faces during data collection:

Fragmented Data 

The data in the healthcare industry comes from a ton of sources, including multimedia, video, pictures, paper, and structured data. Aggregating this data presents a huge challenge for professionals.

On top of that, the data providers are not only doctors or nurses. Public health specialists, patients, and employers also provide data. Thus, there is a risk of duplication, missing information, and inaccurate unification of the information. 

Owing to this, the patient’s journey towards well-being can be delayed or hindered. Sometimes, there’s a lack of communication between all involved parties, leading to the cancellation of certain life-saving or life-improving procedures, resulting in revenue loss and poor resource usage. 

While answering a question about challenges in data analytics in healthcare in an interview, Aswin Chandrakantan, Komodo Health’s Chief Medical Officer, explained how data fragmentation poses a problem. 

He said, ‘’I mean, to summarize it very quickly, there is certainly a data fragmentation problem. I think that there is also a signal-to-noise problem. And to stick into this a little bit deeper, there’s a lot of data, but you need to be able to know where you’re looking for data. You need to know what data elements are relevant. 

You need to know if it’s relevant in all settings of care. You need to know if it’s representative, for you to then say, oh, I can make an informed care choice, or I can make a choice around whether or not I want to engage a provider in a specific way.’’

Furthermore, he explained how collecting the data in a better way can help simplify the decision-making steps for healthcare professionals in all settings. 

‘’And today, it’s been extremely challenging A, because of data fragmentation issues, B, because we’ve lacked the algorithms and the understanding of what we do next with the data that we have, in order to drive activity in the market. 

And so one of the things that I’ve seen is that when teams like medical affairs, commercial clinical development teams when they don’t know what to do, they do what they know. And you tend to revert back to the ways in which things have always been done.’’

In a situation like this, the collaboration of health-tech companies could benefit the industry. Chandrakantan also expressed this in the interview by saying, ‘’And that’s where I feel like Picnic and Komodo, we’re always pushing our clients around, to think differently. Yes. It’s not going to always be easy. Yes. It’s not going to always be necessarily cheap, but it may be the most efficient path, and it might be the one that actually drives the greatest patient, best patient outcome.’’ 

Ever-changing Data 

Another challenge in this regard is the dynamic nature of healthcare data. Physicians and patients move their locations, may progress to an advanced professional level, or retire. Similarly, healthcare organizations may be involved in a merger, relocate to a new place, or be acquired by another group. 

More importantly, new drugs, treatment methods, and personalized care models are introduced every other day. Owing to this, it becomes quite hard to keep healthcare data properly organized and clean. 

To a layman, a change in state data may not seem like a big problem. However, in a healthcare setting, this could lead to inadequate health responses, delayed care programs, and poor patient experience. 

Data Storage 

The healthcare industry is also dealing with problems in terms of storing data. Front-line workers aren’t always concerned with data storage, but they are actively contributing to addition in an organization’s data since they collect this information from the patients. 

Storing large amounts of data costs money, requires additional security and a competent IT team. With a growth in the data volume, the operational costs of a healthcare premise also increase. 

Most healthcare organizations prefer to keep their data on-premise. However, it’s prone to the following problems: 

  • Security compromises 
  • Increased cost of security 
  • Costly to scale on-site server networks 
  • Difficult maintenance
  • Formation of data silos in departments 

As per a survey, 90% of healthcare organizations use a cloud-based IT infrastructure for data storage. Cloud storage reduces upfront costs and allows easy scaling. However, organizations have to be highly careful about the IT companies they partner with since all their practices should be in compliance with HIPAA

To solve this problem, some organizations use a hybrid approach. Thus, some of their data is on-premise while the rest is in the cloud. However, healthcare organizations that adopt this approach must ensure there’s effective communication between each segment so that data can be shared and transferred when required. 

Data Stewardship 

Data collection and storage aren’t the only problems in the healthcare industry. It’s also important to ensure proper stewardship since healthcare organizations are required to keep this information stored for at least 6 years. 

More importantly, the data can be used during subsequent research in the field. Therefore, the healthcare sector needs to have answers ready for certain questions. Who has used this data before? Why was it used? How was it used? 

Who created the data in the first place and why? Since the data gets complex in complicated health conditions, it’s imperative to have metadata. It allows the healthcare industry to replicate former queries, making them useful for scientific studies. 

Chandrakantan also shed light on this by saying, ‘’Our belief is that we want to build a three hundred and sixty-degree view of the patient’s diagnostic and treatment odyssey, in order to learn from the past. I think one of the biggest challenges to date has been that it’s not that the data doesn’t actually interlink; it’s that there has been a lack of incentive or a fearfulness around bringing data sets together. 

And so at Komodo, our thesis is we have a patient master to which you can, which is like a trunk of a tree, and you can attach a lot of metadata objects to it.’’

He also went on to explain how professionals can understand patient demographics and social determinants once the data are grouped properly. 

He said, ‘’You also have the claim data set, which gives you the longitudinal view of that patient. And in working with companies like Picnic, you’re able to add depth to each one of those settings of care, each one of those encounters, and understand specific patient demographics or social determinants of health. 

You have a canonical and representative view of the population that you’re looking to study. So one of the pieces that I would say that we as an industry have not actualized is we’ve not learned from the journeys of the past.’’ 

Fortunately, this data can also be quite helpful in future studies, often eradicating the need for starting everything from scratch. 

‘’Be it commercial, clinical development, we often look at it as, OK, this is a unique patient population and now we need to start from scratch in terms of data generation versus like the thesis that Picnic and Komodo shares are. 

Well, why aren’t you learning from the journey of hundreds of millions of patients that have already been through the US healthcare system and using that to dive, to work on everything from protocol design, to synthetic control arms and virtual registries, to commercial targeting, to indication and label expansion?

So we hope that the journeys of all of these hundreds of millions of patients can be used right here, right now in order to make informed choices in health care.’’ 

Data Reporting 

After completing the query process, healthcare providers have to form a concise report to make the results accessible. However, if the initial steps of data storage and stewardship are plagued with problems, it will have a downstream effect on reporting too. 

Richard Kho, the Chief Commercial Officer of Picnic Health, explains this, ‘’And so if you’re a pharma company, a team that’s working on understanding outcomes, for example, for multiple myeloma patients, to Aswin’s point about fragmentation, you might be getting some data from Medicare, and it’s really delayed and old, or you might be getting some available claims data. 

You might be getting, of course, your own clinical trial or investigator-initiated studies data. You may be getting or even deploying more traditional ways of looking at this, which is running an observational study or a site-based study at huge cost, obviously.’’ 

He further goes on to talk about the role of companies like his and Komodo in simplifying this process. 

He says, ‘’And so if you think about the complexity of multiple myeloma and, of course, many other diseases out there, to be able to see with hundreds or thousands of patients, how many are on which regimen and what were all of the comorbidities, and what were all of the patient characteristics. And then what were the outcomes on top of that? 

And to be able to do that in a timely way using the deepest and best data available, that’s the type of thing that is possible, for example, using Picnic Health combined with Komodo’s capability.’’

Data Security 

Gone are the days when the healthcare industry only had to worry about the safekeeping of the patients’ electronic health records, although it’s extremely important. Nowadays, hacktivism and cyber attacks have increased substantially.

Everyone in the healthcare industry is familiar with the CHS Heartbleed attack. Some hackers hacked the Community Health Systems database to get access to over 4.5 million patients’ information. 

Kho explained how the collaboration of his company with Komodo could reduce the security risks in healthcare settings. He said, ‘’And so the beauty of our approach at Picnic Health is that we work directly with patients. 

They sign a HIPAA consent that includes their request for Picnic health to retrieve data on their behalf, including not only medical records but other types of data about them that are useful for their care. So we’re literally doing this for their care, right, as they get the app on Picnic Health. And then second, they also are given full informed consent under IRB.

So those of you that are familiar with more of the clinical development side of things, we have an independent review board that goes through all of our use cases and approves them. And then, as I said, the informed consent form is provided to the patient. 

And so the patient is fully aware, and we provide education about this as well. It’s not like one of those things where you see a two hundred page document and somewhere embedded in there is something about privacy or security.’’ 


Like every other industry, the healthcare sector also experiences challenges when incorporating big data analytics into its daily operations. 

However, in the words of ARSENAL ADVISORS CEO Frank Dolan, ‘’If we’re going to move closer towards those goals of truly serving patients, I think it’s really important to understand the innovations that could be a part of narrowing that gap between potential and the actual value that we deliver.’’ 

So, as the healthcare industry works towards bridging this gap, it can actually succeed at leveraging technology and analytics to their maximum potential.



More about the executives mentioned in this article


Aswin Chandrakantan, MD, is Chief Medical Officer and Senior Vice President of Corporate Development at Komodo Health

About Komodo Health

Komodo Health believes that smarter, more innovative use of data and analytics is essential for reducing disease burden. We apply artificial intelligence and other advanced data science techniques to our first-of-its-kind Healthcare Map™, which tracks the unique patient journeys of over 325 million patients. We empower a multitude of healthcare stakeholders – life sciences companies, healthcare payers and providers, patient advocacy groups, and others – to create a more cost-effective, value-driven healthcare system. For more information, visit

Richard Kho is the Chief Commercial Officer at PicnicHealth

About PicnicHealth

PicnicHealth is a healthcare technology company that partners with patients, porting their complete medical records into an easy-to-use online application. The platform gives patients unprecedented access to and control over their medical records and, with their consent, the opportunity to contribute this valuable data to further scientific research. Founded in 2014 by Noga Leviner and Troy Astorino, the company partners with several of the world’s largest biopharma companies and academic research institutions. PicnicHealth has raised more than $35M from Felicis Ventures, Amplify Partners, Foresite Capital, Refactor Capital, and Y Combinator. Learn more at

Frank F. Dolan is the Founder and CEO at ARSENAL ADVISORS


ARSENAL ADVISORS serves the life science industry by shaping and sharing best practices so companies can acquire the capabilities to accelerate innovation and maximize brand potential. ARSENAL creates content and experiences that connect the people and ideas transforming biopharma and medtech. Learn more at