Which of the Following Likely Needs to Be Improved About the New Process to Review Vital Signs?

• Journal List
• BMC Med Inform Decis Mak
• v.xvi; 2016
• PMC4893236

BMC Med Inform Decis Mak. 2016; sixteen: 61.

How to better vital sign data quality for utilise in clinical determination support systems? A qualitative study in nine Swedish emergency departments

Niclas Skyttberg

Capio St Görans Hospital, 112 81 Stockholm, Sweden

Health Informatics Centre, Department of Learning, Informatics, Management, and Ideals, Karolinska Institutet, 171 77 Stockholm, Sweden

Joana Vicente

Wellness Informatics Centre, Section of Learning, Informatics, Management, and Ideals, Karolinska Institutet, 171 77 Stockholm, Sweden

Cambio Healthcare Systems, Stockholm, Sweden

Rong Chen

Wellness Informatics Centre, Department of Learning, Information science, Direction, and Ethics, Karolinska Institutet, 171 77 Stockholm, Sweden

Hans Blomqvist

Section of Anaesthesia and Intensive Intendance, Karolinska Academy Infirmary, 171 76 Stockholm, Sweden

Sabine Koch

Health Informatics Centre, Department of Learning, Informatics, Management, and Ethics, Karolinska Institutet, 171 77 Stockholm, Sweden

Received 2015 Sep 28; Accepted 2016 Jun 1.

Abstract

Background

Vital sign data are important for clinical decision making in emergency care. Clinical Conclusion Support Systems (CDSS) have been advocated to increase patient condom and quality of care. However, the efficiency of CDSS depends on the quality of the underlying vital sign data. Therefore, possible factors affecting vital sign data quality need to be understood.

This report aims to explore the factors affecting vital sign data quality in Swedish emergency departments and to determine in how far clinicians perceive vital sign data to be fit for employ in clinical determination support systems. A further aim of the study is to provide recommendations on how to improve vital sign data quality in emergency departments.

Methods

Semi-structured interviews were conducted with xvi physicians and nurses from 9 hospitals and vital sign documentation templates were collected and analysed. Follow-up interviews and process observations were done at three of the hospitals to verify the results. Content analysis with constant comparison of the data was used to analyse and categorize the collected data.

Results

Factors related to care process and information technology were perceived to affect vital sign data quality. Despite electronic health records (EHRs) beingness available in all hospitals, these were not always used for vital sign documentation. Only four out of nine sites had a completely digitalized vital sign documentation menstruum and paper-based triage records were perceived to provide a better mobile workflow support than EHRs. Observed documentation practices resulted in depression currency, completeness, and interoperability of the vital signs. To amend vital sign information quality, we propose to standardize the care process, improve the digital documentation back up, provide workflow support, ensure interoperability and perform quality control.

Conclusions

Vital sign data quality in Swedish emergency departments is currently not fit for employ by CDSS. To address both technical and organisational challenges, we suggest five steps for vital sign data quality improvement to be implemented in emergency care settings.

Electronic supplementary material

The online version of this article (doi:x.1186/s12911-016-0305-iv) contains supplementary material, which is available to authorized users.

Keywords: Computer-assisted decision making, Information quality, Electronic wellness records, Emergency care, Medical computer science, Vital signs

Groundwork

Vital sign data are important in emergency care decision making, especially for prioritization and identification of severe illness. To screen for sepsis, vital signs are essential in early detection [1, ii], and it is well known that rapid detection and early handling are central factors to improve patient survival [three]. Studies in the emergency section testify that vital signs can exist used in predicting cardiac arrest [iv, five] and sepsis outcome [six]. Further, the vital signs are broadly used in clinical care in the calculation of alert scores that are aiming at predicting clinical deterioration [vii, eight]. In emergency care, the first use of the vital signs is often in the triage of arriving patients [9–11]. Reports show that triage systems are used in virtually all Swedish emergency departments [12, 13] and that the majority of the hospitals utilize the rapid emergency triage and handling system (RETTS) [10]. The common denominator of most triage models is that they use vital sign measurements to calculate a triage score.

Clinical Decision Support Systems (CDSS) using data from electronic wellness records (EHR) are advocated to amend clinical upshot [fourteen, xv], quality and patient safety [16]. In emergency care, some studies take indicated that triage CDSS may provide reliable calculations of triage severity [17] and fifty-fifty improve risk cess [18], while others have questioned the readiness for automation [19] . To be trustworthy the CDSS need to provide reliable communication to the clinician. The quality of the CDSS recommendations will depend on the quality of the underlying data in the EHR [20]. When comparing to traditional definitions of data quality, stating that quality is adept enough when the data tin fulfil the intended purpose [21–23], this means that the underlying data has to exist able to support reliable calculations in the CDSS. To provide reliable triage scores, the vital sign data needs to exist right, complete, and timely available. A comprehensive review past Weiskopf and Weng describes these three principal information quality categories every bit; Definiteness, Completeness, and Currency [24], where the correctness of the data indicates to how true the documented vital signs are, completeness refers to whether all expected vital signs are actually registered in the EHR and currency is linked to the temporal aspects of the documented vital signs.

The yearly report on healthcare It and e-health in Sweden [25] shows that EHRs have a penetration of a 100 % in Swedish emergency care hospitals, and based on this high penetration combined with the broad use of triage systems, a high completeness of vital signs may be expected. However, some studies indicate problems with vital sign data quality. A written report by di Martino et al. testify that the completeness of the vital signs needs improvement [26], farther Genes et al. [27] land that vital sign data definiteness is relatively depression, and finally Ward et al. [28] question the operational integrity of the time stamps after EHR implementation. Less is known about which factors affect emergency care EHR data quality and in what way they impact it [26, 29]. The di Martino [26] study shows that a clinical comeback program may increase completeness and Wager et al. [29] country that the entry device will take an effect on correctness and currency. If CDSS is expected to have an bear upon on patient rubber and quality, more than knowledge is needed on how to assure and improve clinical information quality. This cognition tin exist used both to back up CDSS development and in clinical quality improvement work.

Objective

The aim of this report is three-fold. Firstly, to explore the factors affecting vital sign information quality during measurement and documentation in Swedish emergency care. Secondly, to determine how far clinicians perceive documented vital sign data to be fit for use in clinical conclusion back up systems. Thirdly, to provide recommendations on how to better vital sign information quality in emergency care.

Methods

We explored the factors affecting vital sign data quality in emergency intendance using a qualitative approach. The explored process of vital sign collection and apply is described in Fig.1. Data collection was done through semi-structured interviews, observations and analysis of documentation templates in nine purposefully selected Swedish emergency departments. We used content analysis with constant comparison to categorize the information [30–32].

Emergency care process. Vital sign measurement and documentation in the emergency care procedure

The written report was performed during a period of x months (August 2014 – May 2015).

Setting and participants

Inclusion criteria for the participants were a degree equally Medical Doctor (MD) or Registered Nurse (RN) with a minimum of 5 years of experience in emergency care, in detail, triage and vital sign documentation. Quality managers at the sites were contacted and helped to identify a total of sixteen participants that fulfilled the inclusion criteria. The participants were invited by e-mail and all accepted to participate in the written report.

Sites were included aiming at variability in size and regional distribution. In total, nine hospitals (Tabular array1) were included in the written report, five university hospitals (UH) and four secondary referral centres (SRC). Three different Electronic Wellness Record Systems were used at these sites.

Table 1

An overview of the performed interviews

SiteNumber	Type	ED Visits	#MDs Interviewed	#RNs Interviewed
Site 1	UH	65000	1	1
Site 2	SRC	40000		3
Site 3	SRC	30000	1
Site 4	UH	97000		2
Site v	UH	53000		1
Site 6	SRC	80000		1
Site seven	SRC	39000		3
Site 8	UH	45000		2
Site 9	UH	53000		1

Data collection

Data was collected in three subsequent steps. Showtime, xvi semi-structured interviews were conducted at nine different sites (Tableone). The interviews were performed using a semi-structured interview guide (included in the Additional file ane) covering different aspects of data quality such as abyss, correctness, and currency. The interview guide besides aimed to cover how the participants experienced existing vital sign information quality and perceived opportunities to increment vital sign information quality in the EHRs. The guide was tested in a pilot interview. All interviews were voice recorded, performed in Swedish, either on-site or by phone, and lasted for virtually thirty min.

Confirmation was sought with the participants to verify the findings from the interviews. This was done through 2d round interviews with open-ended discussions on the themes and categories found in the results. The participants were selected equally they represented dissimilar documentation practices in the initial interviews. The interviews were performed on site in Swedish, lasted for about 45 min each and were voice recorded.

After the interviews, observations were performed at three selected sites aiming to complement the information collected during the interviews. Sites No. 2, 4 and 8 were purposefully selected for the observations as they represented examples of three different documentation practices that were institute in the initial interviews (Table3). Observations focused on vital sign measurements and documentation in the emergency departments, aiming to encompass a variation in locations, staff, and clinical situations. The observations aimed to confirm the findings from the interviews in clinical practise and they lasted about sixty min per site. The observations were performed past the researchers together with a nurse from the site. The observers were free to move around the emergency department during the observations and during the observations photos and field notes were taken and samples of documentation templates were collected from the emergency departments. From the observations, a structured report was written. The ascertainment study protocol is included in the Boosted file ane.

Table 3

Documentation practice

Documentation do	Description	Number of sites
Paper-based documentation	Documentation on a structured paper-based template and later on scanned into the EHR in pdf format. No entries of vital signs were done in the EHR.	2 Sites
Mixed Documentation	Washed on a newspaper-based template and after transferred into a digital EHR template.	3 sites
Digital documentation	Documentation on a digital template	4 sites

Two researchers (NS and JV) performed the interviews and observations. A tertiary and fourth researcher (SK and RC) continuously gave feedback on interviews and results.

Data analysis

The recorded interviews were transcribed and coding was washed past reading through the transcripts. Quotes and meaning units were translated into English. The codes were inductively abstracted into categories and themes.

Transcription, coding, brainchild and rechecks with the sound recordings were done continuously. Ii researchers (NS and JV) performed coding and abstraction separately. Discussions in the research group were held to compare coding and emerging categories, making concepts and categories change and evolve during the procedure. Changes in the coding and categories macerated over time and eventually a consensus on categories was reached in the enquiry grouping. Interviews were performed until no new concepts or categories emerged. Subsequently consensus and saturation, the results from the analysis were confirmed by 3 of the initial participants in the form of open up second-round interviews. These interviews were transcribed and coded in the same manner as the showtime-round interviews. Additional data such as documentation templates and field notes from the observations were later on included in the analysis. This further confirmed saturation and strengthened our confidence in the interpretation of the initial interviews.

Ethical considerations

Ethical approval was practical for at the Stockholm Ethical Committee simply not considered to be needed (Dnr 2014/1207-31/four). Data on the study was given in advance and informed consent was obtained. Participation was voluntary and confidentiality was bodacious. To clinch confidentiality none of the quotes are connected to the sites in the publication.

Results

Factors affecting vital sign information quality

The factors resulting from the content assay are presented according to the main themes and categories that were constitute in our analysis (Table2). The table includes examples of subcategories, meaning units and corresponding quotes. The quotes are related to the type of documentation practice found at the site; newspaper-based documentation, mixed documentation, and digital documentation (Table3).

Tabular array 2

Factors affecting vital sign data quality

Themes, categories and instance quotes
Theme	Main category	Subcategory	Meaning unit (examples)	Quote and blazon of documentation practice
Intendance process	Standardized process	Standardized triage	Standardized Triage - Securing Vital Sign measurements	"We practise triage on all patients arriving at the emergency department. No divergence if they are arriving past ambulance or walking in. A brusque history and vital sign measurements are included in all patients." PD
		Standardized documentation	Standard of documentation improves completeness	"I think it has improved a lot (data quality of vital sign). Before the structured workflow was set, respiratory rate was not completed as frequently every bit today." DD
		Failure to comply	Failure to comply - Private Clinical Sentence	"If a patient has a minor complaint the standard may not be experienced equally relevant. In those cases, there may be failure to comply" DD
		Lack of standard	Lack of Standard in repeated measurement documentation	"A patient was kept close to the nurse desk-bound with automated continuous vital sign measurements for hours. Only ii recordings were entered into the EHR." DD
	Direction	Quality command	Government Control of Care Quality	"We received feedback from the National Board of Health and Welfare because our documentation of vital signs. That has fabricated u.s.a. change routines on documentation and the way we follow up on compliance with documentation standards" DD
		Change management	Resistance to modify - switching to digitalized menses	[switch to digitalized flow] "It wasn´t completely easy to achieve. At first, the physicians lacked the paper. But nowadays no i wants to switch back." DD
		Education/training of staff	Understanding of documentation importance	"You have to educate to increase the understanding why it [documentation] is important. Otherwise, there may be neglect of registrations." Dr.
	Competence and knowledge	Method and equipment	Error sources - temperature, ear wax	"When it comes to temperature measurements at that place may be problems due to unproblematic fault sources, like wax in the ear canal." Medico
		Clinical competence	Clinical Validity check	"You cannot e'er trust the device. You take to make a clinical validity check. If at that place is a trouble, you may take to recheck or alter method." DD
Information technology	Workflow support	Mobility	Mobile documentation required when switching to digitalized flow	"Unless we get access to computers at every room or more than mobile ways of working, like iPADs we will likely hold on to the newspaper triage record" PD
		Overviews	Overview of vital sign measurements	"Nosotros demand a good overview of measurements so that they can be followed over time." PD
		Checklists	Process overview and checklist.	"What we lack in the EHR is a usable culling to paper-based triage tape. Information technology should provide overviews and checklists to make sure that everything that should be done gets done and that goose egg is forgotten" PD
		Calculation support	Automatic calculation of triage score	"We enter the curt history and vital signs in the EHR and with a click, the triage colour volition exist calculated." DD
	Documentation support	Structured documentation	Documentation templates - feet about forgetting	"It makes sure that everything gets done and that nosotros all do it the same style. It will decrease anxiety about forgetting. "PD
		Logical controls	Logical controls - dictation and complimentary text	"We apply dictation to enter the vital signs into the EHR. It will be entered in gratuitous text. There are no congenital-in logical controls." MD
		Completeness checks	Completeness checks	"To complete the triage all vital signs have to exist registered. Information technology is a role of the triage process and the arrangement demands a full set." DD
		Automated registrations	Automatic registrations of measurements to improve completeness	"Automatic registration of repeated measurements would really improve documentation. If patients are measured every 15 min, in that location is no time to manually annals all measurements." DD
	Interoperability	Interoperability within system	Reuse of information between modules in EHR	"Nosotros are working in our acute care module. We don't want to use separate parts of the arrangement making double entries" DD
		Interoperability between system	Sharing data with pre-hospital records	"Vital signs will be measured in the ambulance. We volition manually copy them into our EHR." DD

Footnote: In the table the following abbreviations are used in relation to the quotes; Newspaper-based documentation (PD), Mixed documentation (Dr.) and Digital documentation (DD)

Intendance process

A standardized process

The interviews showed a perceived importance in following a predefined workflow to increase the quality of vital sign data measurement and documentation. Without a standardized process, individual staff considerations guided decisions nigh when and how to mensurate and document vital signs. Hence, a standardized process was perceived to increase quality by reducing individual variations.

The results showed that virtually patients were expected to be triaged early after inflow at the emergency department. "Nosotros practise triage all patients arriving at the emergency section. No difference if they are arriving by ambulance or walking in. A short history and measurement of the vital signs are expected in all patients." (digital documentation). All but one site used the rapid emergency triage and treatment organisation (RETTS) as the triage system and usually, triage was expected to be completed inside 15 min of arrival. Even so, at 2 of the sites triage was just performed if there was a waiting time to see the md. If at that place was no waiting time, vital signs were measured at the physicians' initiative or order only.

Switching to a standardised workflow, where all patients were expected to be triaged, was mentioned to improve the measurements and documentation of vital signs. As discussed on the example of respiratory rate in one interview; "I remember it has improved a lot (data quality of vital sign). Before the structured workflow was prepare, respiratory charge per unit was not completed every bit frequently as today" (digital documentation).

Repeated measurements of vital signs were mentioned as an expanse with poor data quality. This was attributed to the lack of a standardized procedure "There are difficulties in finding a routine both in measurement and documentation (in the re-evaluation of vital signs)" (digital documentation). One interpretation of the divergence between triage and repeated measurements is that the showtime of the emergency intendance process is easier to standardize. The after part of the procedure may be more diverging and depending on the patient'southward complaints (see besides Fig.1). Ane of the sites mentioned having implemented vital sign rounds at the emergency department. Patients with high triage scores were expected to have their vital signs rechecked every xv to 30 min. The aim was to avert unnoticed patient deterioration by standardizing reevaluation of the vital signs.

Fifty-fifty with a set standard in place not all staff have and follow the routine; "If the patient has a minor complaint the standard may not ever be experienced as relevant. In those cases, at that place may be a failure to comply" (paper-based-documentation). Private decisions were described to touch the vital sign data as well as the quality of the care given. In one of the interviews, this blazon of individually based triage was described "the most dangerous of triage practices" (digital documentation). However, some of the interviews mentioned that this do may exist more mutual among experienced staff. If a witting conclusion to diverge from the standard was made by experienced staff members, this was perceived as having less impact on the quality of care. The event on the vital sign data quality would be the aforementioned regardless of staff experience.

A well-defined workflow was experienced to increase the number of complete vital sign measurements. The triage procedure was an case used in all interviews. Yet, to exist fit for apply in CDSS the vital signs too had to exist registered in the EHR, as is further discussed under documentation support.

Management

Direction factors were found to impact the data quality of the vital signs. Opportunities for a quality increase were seen by decision-making the quality and giving feedback on performance, but likewise by leading the organization through resistance to modify. When discussing quality management both local and government control were mentioned as important. "We have done manual record reviews to check the documentation of vital signs. We practise regular sample checks of vital sign registrations and give feedback to the staff" (mixed documentation). "Nosotros received feedback from the National Board of Health and Welfare considering our documentation of vital signs. That has fabricated united states of america modify our routines on documentation and the mode we follow upwardly compliance to the documentation standards" (digital documentation). The observations supported that feedback on quality was given, for example, quality indicators were found posted on boards in staff areas. Analysed documents showed that feedback focused on comparison results of the measured indicators to set goals.

Change management referred to resistance to change in an organization. "No, nosotros don´t have whatsoever data on triage vital signs in the EHR as we keep that record on paper. It is tradition and routine and nosotros are quite stuck with it." (paper-based documentation). At some of the sites such resistance had been overcome "It wasn´t completely easy to reach (switch to electronic documentation), but nowadays no one would consider moving dorsum to paper" (digital documentation). These sites stressed the importance of leadership and direction in making the change happen.

Competence and noesis

Understanding the medical equipment, measurement methods and being able to critically evaluate the results was perceived to affect the definiteness of the documented vital signs. "Because temperature measurements there may be bug due to elementary fault sources, similar wax in the ear" (mixed documentation). If the limits and error sources were not understood and known this was perceived equally leading to incorrect registrations of vital signs.

The importance of clinical competence when evaluating the plausibility of vital signs and the knowledge of error sources in measurement methods were mentioned as related to data quality. This was discussed in relation to the correctness of the registered vital signs. "Yous cannot always trust the device. You lot have to make a clinical validity check. If there is a problem, you may take to recheck or change method." (digital documentation). The competence and cognition factors were experienced to be managed past training of the junior staff and by the support given by more experienced colleagues.

Information technology

Documentation support

The interviews showed that three dissimilar documentation practices were in place at the sites (Tabular array3), and only four of the sites did apply a fully digitalised documentation period of vital signs. All sites had implemented EHRs but these were not perceived every bit supporting documentation in a mobile fast paced emergency care context where staff and patients were described as mobile but IT systems were experienced to be stationary and not always bachelor to the staff at the point of care. Newspaper-based records were described as light-weight, portable and easy to apply when recording vital signs. "We use the paper record equally an emergency record, we register all vital sign measurements and past that fashion, they are shut at mitt without any need to log in to the estimator" (paper-based documentation). When paper-based templates were used and transferred to the EHR the currency of the data might exist afflicted. Even if time stamps were reported to be manually set in the EHR to correspond measurement time, this do was described as inconsistent amid staff especially when the workload was loftier.

Retrospective documentation was also perceived to touch the willingness to reenter data into the EHR. "It feels similar double documentation and double piece of work to transfer the paper documentation into the electronic health record." (paper-based documentation). When transferring information to the EHR not all measurements were expected to be registered. At sites where documentation was fully digitalized, according to the interviews, the staff was observed using paper notes to record the values in order to enter them into the computerized record afterwards. This observational finding contrasted findings in the interviews, where all the documentation workflow was described as done in the EHR.

The use of specific documentation templates for the emergency care procedure was expected to support right and complete registrations but none of the sites was observed to provide abyss checks with reminders or other intrusive ways to force completeness of vital sign documentation. The EHRs were observed to give plausibility warnings on values that were out of biological range but for these warnings to piece of work standardized templates had to be used. Automation of documentation was mentioned in the interviews as a possible way of affecting completeness and currency. "Automatic registration of repeated measurements would really improve documentation. If patients are measured every 15 min, there is no time to manually register all measurements." (mixed documentation). No sites had any information commutation in place between of measurement devices and the EHR.

The documentation support was discussed under all aspects of data quality, and mobility and ease of use were found of import to support timely and consummate registrations of the vital signs. The use of documentation templates for the emergency intendance process continued the documentation support category to the workflow support and interoperability categories.

Workflow support

Entering vital signs into the EHRs was not perceived to back up the intendance process but rather linked with subsequently reuse of information. "The documentation (in the EHR) may be important as a reference after at the ward" (mixed documentation). "In my piece of work at the emergency department the documentation (in the EHR) is not important for the acute care process … I rely on the newspaper-based triage tape" (mixed documentation). This lack of workflow support was experienced to decrease timely and complete registrations. When discussing expectations on a support that would increment the quality of the registered vital signs, three sub-categories were mentioned. These were overviews of information, reuse of information and mobility in the workflow.

Overviews of information were perceived as important for workflow support. Such overviews included read and write functionality combined with a checklist to back up complete registrations of the vital signs. "(in the EHR) we lack usable functionality that is in that location in the paper triage record. Information technology provides an overview and a checklist of of import information and makes united states remember things that are supposed to be done." (paper-based documentation). If the information entered in the EHR was reused in the process at the emergency department willingness to enter information was expected to increment. Examples given of such reuse included automatic calculation of triage or warnings scores.

A perceived lack of workflow support in the EHRs was discussed in many of the interviews, but the statements were conflicting, and both at sites using paper and digital documentation there were participants acknowledging good-plenty workflow support in the nowadays EHRs. These participants further stated that management was the key cistron to fully implement the EHRs. Yet, the overall impression from the interviews and observations was that the EHRs used at the emergency departments today exercise not fully support a workflow resulting in current and complete registrations of vital signs.

Interoperability

The EHR systems were described as complex with split up modules using different keywords for similar concepts. The lack of terminological binding was described to hinder reuse of information within a system. The staff was reluctant to add multiple entries and preferred to use parts designed for their procedure. "We are working in our acute care module. We don't want to use separate parts of the system making double entries" (mixed documentation).

Sample documents and observations verified that multiple keywords were used for documentation of vital signs and that keywords lacked both binding terminologies and differed in underlying data types. Data could be entered as free text in one keyword and as a numerical value in another. In some sites using paper-based templates for triage, the template was scanned and stored every bit an paradigm in the EHR and such information was not reusable in digital systems. These examples showed that even though information exists inside a organisation it may not exist available to CDSS.

The interviews discussed that separate systems were used in the emergency care flow. The pre-hospital squad used a digitalised organisation to register vital signs but those registrations could not be retrieved or reused by the hospital EHR. As described in one of the sites using digital documentation; "Vital signs will exist measured in the ambulance and registered in their system. We volition manually copy them into our EHR" (digital documentation). These findings were confirmed by the observations.

Every bit a registration that was not retrievable by the CDSS was perceived equally incomplete, the interoperability category was found to be continued to the abyss of registrations. Interviews and observations showed that interoperability of the vital signs was expected to be low, making the vital signs so difficult to retrieve that they would be considered unfit for utilize.

Fettle for use

The experience of the participants was that the vital signs registered in the EHR would not exist fit for use for calculation of warning scores or triage scores. The findings showed that 5 out of 9 sites documented the vital signs on a paper record and that the paper-based documentation was connected to a low completeness of in the EHRs. Although some sites would transfer the registrations to the EHR in that location would likely be a delay earlier the vital signs would be available. The fourth dimension to registration was hard to guess from the interviews and described equally variable from minutes to hours, depending on workload and private preferences. At sites that were supposed to utilize a fully digitalized documentation flow, the staff was observed to write down measurements on newspaper to keep for a later on registration in the EHR, and this supported the interview finding that some delay was expected at all sites. The main upshot of the documentation practice was perceived to impact completeness and currency of the vital sign registrations in the database. The correctness of the vital signs was perceived to be a modest problem. Although error sources were discussed, these were not expected to lead to frequent incorrect registrations of vital signs.

To be fit for use in CDSS the information has to exist shared within and between IT systems and work processes. The concept of Interoperability relates to how fix the vital signs are for generic reuse betwixt Information technology systems and work processes. It includes the functional view of interoperability co-ordinate to Kubicek [33] that is technical, syntactic, and semantic and business process interoperability.

The results show that although information does exist within the EHRs it may still be very hard to connect and integrate it into CDSS. Fifty-fifty if the vital signs were correct, complete and current they were even so not considered fit for use equally they lacked binding to standardized terminology or information models. This was further supported in gathered templates and in the observations, where the findings of multiple keywords, with differing data formats, and lack of terminology binding showed a low semantic interoperability. These findings show that registered vital sign data, in Swedish emergency departments EHRs, are likely to be unfit for use in CDSS due to lack of completeness, currency, and interoperability.

Opportunities to improve quality

All of the interviews included discussions on how to ameliorate vital sign information quality. Summarising these discussions pb united states of america to a five-step approach for vital sign data quality improvement.

1. Standardize the intendance process

   Following a standardized process was experienced to be of importance to meliorate completeness. The triage part of the emergency care process was mentioned to be standardized by all sites, but a standardized re-evaluation of vital signs was mostly experienced every bit a challenge. This was explained past the many different atmospheric condition investigated and treated at the emergency section. One of the sites mentioned they had started vital sign rounds where they aimed to recheck and document vital signs every 15–30 min on patients with a triage score indicating loftier acuity.

2. Improving digital documentation support

   Electronic health records were available just not used for documentation of vital signs at all sites and switching to a digital flow was experienced to meliorate the completeness of vital signs in the EHR. This switch had been made in 4 of the sites and although hard to brand, the switch was perceived every bit worthwhile and once made broadly accustomed. However, manually registered vital signs in the EHR were not experienced to define timely registrations. Integration of medical devices and transfer of information were discussed in all interviews as possible ways to improve documentation of complete and current vital signs.

3. Provide workflow support

   The findings showed that the EHRs were non perceived to provide the aforementioned lightweight, easy to apply, workflow support as the paper-based triage records, and further the EHRs were non experienced to evangelize a sense of usefulness of the registered vital signs. These factors were establish to impact the abyss and currency of the vital signs. To further improve vital sign data quality system developers were encouraged to develop mobile solutions that focus on the back up of the emergency department workflow.

4. Ensure interoperability

   The findings evidence that the EHR systems are non set to exchange vital sign data. The results showed that different keywords and templates without terminological bounden or standardized reference models were used. To ensure that entered vital signs were possible to re-use at that place was an experienced need for improved interoperability.

5. Perform Quality Command

   Giving feedback on data quality and was experienced as a mode to better completeness. Management focus on data quality was likewise perceived to serve an educational purpose equally discussions on quality and possible error sources were thought to increase staff understanding of the importance of documentation.

Give-and-take

This written report shows that a number of factors impact the vital sign information quality in the emergency intendance procedure. The main themes included factors related to the care procedure and information technology, and amid the care procedure factors information technology was considered important to follow a set standardized process to minimise individual variation. Direction factors were constitute important in making the transition to a digitalised documentation flow, but that factors within the it theme could facilitate or hinder the apply of the provided systems. To facilitate utilize, the EHRs were encouraged to provide workflow and documentation support directly aiming at the emergency care process. Due to the lack of usability, the overall experience is that the vital signs will not be fit for use in CDSS due to low abyss, currency and lack of interoperability.

Sweden is an early adopter country when it comes to wellness Information technology and different types of systems such as laboratory data systems, picture archiving and advice systems and also EHRs were introduced earlier in Sweden than in many other countries. Today the penetration of EHRs in infirmary care is 100 % and the number of PC clients is about 1 per employee [25]. In this context, the conditions for achieving loftier data quality through indicate-of-care documentation seem to be platonic. Withal, in our study only four out of nine hospitals had completely digital registration of vital signs. Despite the loftier coverage, EHR systems do not seem to be fully used in the emergency intendance setting. Fifty-fifty though EHR systems provide emergency intendance modules with triage back up, there is meaning resistance when switching to a fully digital workflow, and this type of barrier to the use of EHRs is well described in other studies [34, 35]. According to our results the low usage is related to usability factors, equally the electronic wellness records in the market today are not perceived to provide the mobile workflow back up that is nowadays in the paper-based triage records. We conclude that although the systems provide generic EHR support for health care there seems to exist an experienced lack of process and workflow support in emergency care.

Our results show that fifty-fifty with a switch to digital registrations, a paper-based documentation may persist that will cause delays in registration and hinder the reuse in real-time CDSS. This finding, indicating problems with the currency of the data, is in line with other studies. Ward et al. studied the effect of the transition from paper-based emergency care records to EHR and found that there are problems with fourth dimension stamps of the information [28], and farther a written report on blood pressure measurement registrations in emergency departments also indicated issues with currency [36]. If the systems used for documentation are not bachelor to the staff at the indicate of care registrations will be delayed [37, 38]. We would argue that the point of care documentation concept would demand to include mobility as patients and staff in a busy emergency department are on the motion. Stationary workstations were experienced to hinder timely data drove and limit availability and admission to information due to login issues. To stimulate complete and timely vital sign registrations EHRs and CDSS were expected to provide workflow back up with a direct sense of usefulness [39, 40]. The results show that adding back up on triage scores and overviews targeting the patient data related to the emergency care process could increase the usability. Without this support the documentation may be regarded as retrospective and the sense of importance may be afflicted.

The results of this written report show that the EHR vital sign data may be complete, correct and current but still very hard to brand available and connect to a CDSS. The importance of interoperability to realize the potential of EHRs and CDSS is well described in bookish publications and regime reports [41, 42]. Furthermore, this study shows that interoperability of the data needs to exist discussed when considering data quality, especially when the utilise case is including reuse of data in CDSS. The utilise of standardized templates may assist increasing semantic interoperability within the EHR [43, 44]. By keeping vital signs registered under specified keywords and consequent data formats the information volition be easier to recall and connect between systems. Preferably these keywords should be based on reference EHR models and connected to standardized reference terminology and coding systems [45].

Interoperability with medical devices is considered as a style of increasing consummate, current and connectable registrations [45, 46]. If medical devices are able to deliver vital signs into the EHR the delay of entry may decrease thereby increasing the currency. The integration of medical devices is as well likely to increment completeness as this study shows that repeated measurements of vital signs are not registered in the EHR. It should be emphasised that automatic registrations and conclusion back up must be used in accordance with the clinical exercise in order to improve medical quality and safety. If not used with added clinical knowledge and feel, automatic registrations and CDSS may acquit a take a chance of unreflective and not patient-centred practice.

Limitations

There are a number of limitations to the study. Firstly, as only experienced staff was included this may cause omission of areas specific to inexperienced staff. The transferability of our results is dependent on the sample of participants and past the fact that the results are based on expressed individual experiences. To strengthen the transferability of the results we included a representative sample of both university hospitals and secondary referral centres, using iii different EHRs and representing three different piece of work practices. Secondly, the researcher's knowledge and feel with Swedish emergency departments and CDSS may predispose to conclusions in the analysis. To counter the effects of predispositions within the research group the data analysis has been continuously peer-reviewed. Thirdly, the reliability of the results over time will be affected by the limited fourth dimension frame of the study and also past the fact that the results are dependent on present workflows and presently used Information technology systems.

Implications for clinical practice, EHR/CDSS research and evolution

Further quantitative research may need to confirm our results regarding vital sign information quality in emergency departments. When studying the vital sign information quality, the aspect of interoperability has to be taken into business relationship. Data cannot be considered fit for utilize if it is not possible to retrieve and connect between It systems without pregnant work on mapping. Completeness may be of special interest in the emergency care context every bit quality is experienced low regarding repeated measurements. Time stamps and the currency of the information should also exist a focus point. The results from this written report show that there may exist significant quality deficiencies in the currency of the information.

If clinical practice is to do good from CDSS, information technology is essential that the documentation period of vital signs is digitalized. In clinical practise digital templates for vital signs based on standardized reference models with terminology binding need to and can exist developed in current IT systems by system administrators and clinical staff. EHR system developers should focus on delivering mobile workflow support within EHRs. If to be implemented in emergency care, CDSS likely demand to assure the collection of high-quality vital signs as existing data may non be fit for utilize.

Decision

This report shows that standardization of the workflow is an important concept when discussing vital sign data quality in Swedish emergency departments. A well-defined workflow including measurement and documentation is experienced to reduce individual variation and increase quality. Withal, to make sure that the documentation is digitalized, data technology has to provide adequate documentation back up, otherwise paper-based documentation will be favoured. Lack of such adequate support was described in all of the interviews. This may be an important gene why only four out of the nine sites used the EHR to certificate vital signs. Because the EHRs, although present at all sites, were non used to register complete and timely vital signs, the information quality was not perceived to be fit for apply in calculation of alert scores. Based on these finding we discuss a v step program to amend vital sign data quality.

Abbreviations

CDSS, Clinical Decision Support Systems; EHR, Electronic Wellness Record; Dr., Medical doctor; RETTS, Rapid emergency triage and handling arrangement; RN, Registered nurse; SRC, Secondary referral centre; UH, University hospital.

Acknowledgements

Monica Rådestad, Måns Belfrage, Tomas Leijon, Simon Askelöf, Bo Orlenius, Göran Karlström, Styrbjörn Östberg och Bengt Öberg have all contributed to making this study possible.

Funding

No external funding has been obtained to perform or support this study.

Availability of information and materials

To assure confidentiality of the participants further information from the interviews will not be shared.

Authors' contributions

All authors contributed to the work in a way that fulfils all of the four ICMJE criteria for authorship, including reading and approving the final manuscript. NS was the project leader, coordinated the report and was involved equally in all parts. JV contributed to preparing, performing and transcribing interviews and analysis of results. RC contributed to the assay of the results, drafting and reviewing of the manuscript. HB supported the work past feedback on results and analysis and critical reviewing of the manuscript. SK acted equally project possessor of the written report contributing with scientific methods, drafting and reviewing of the manuscript. All authors read and approved the last manucripts.

Authors information

Niclas Skyttberg (NS), MD, CMO Capio St Görans Sjukhus, Ph.D. pupil at LIME, Karolinska Institutet.

Joana Vicente (JV), DDS, MSc – Health Informatics, LIME, Karolinska Institutet and Cambio Healthcare Systems. Participated in the report as a master thesis student and was after employed by Cambio.

Rong Chen (RC), MD, Ph.D., CMIO Cambio Healthcare Systems and affiliated Researcher at Department of Learning, Computer science, Management, Ethics, Karolinska Institutet.

Hans Blomqvist, MD, Ph.D., Section of Anaesthesia and Intensive Intendance, Karolinska University Hospital.

Sabine Koch (SK), Professor of Health Informatics, Managing director of Health Informatics Centre, Department of Learning, Information science, Management, and Ethics, Karolinska Institutet.

Competing involvement

NS is employed as Chief Medical Officer at one of the study sites. RC is employed equally the Principal Medical Computer science Officer at Cambio Wellness Care Systems. JV is employed by Cambio Healthcare Systems. Cambio Health Care Systems is an Electronic Health Record provider to five of the studied sites.

Consent for publication

Consent for publication was obtained together with the consent to participate.

Ethics approving and consent to participate

Ethical blessing was practical for at the Stockholm Upstanding Committee just not considered to exist needed (Dnr 2014/1207-31/4). Information on the written report was given in advance and informed consent was obtained. Participation was voluntary and confidentiality was bodacious. To assure confidentiality none of the quotes are connected to the sites in the publication.

Additional file

Additional file ane:^{(13K, docx)}

Interview guide and observation protocol. The interview guide and ascertainment protocol used in the report. (DOCX 13 kb)

References

ane. Kenzaka T, Okayama M, Kuroki Southward, Fukui M, Yahata S, Hayashi H, Kitao A, Sugiyama D, Kajii E, Hashimoto G. Importance of Vital Signs to the Early Diagnosis and Severity of Sepsis: Association between Vital Signs and Sequential Organ Failure Assessment Score in Patients with Sepsis. Intern Med. 2012;51:871–876. doi: 10.2169/internalmedicine.51.6951. [PubMed] [CrossRef] [Google Scholar]

3. Dellinger RP, Levy MM, Rhodes A, Annane D, Gerlach H, Opal SM, Sevransky JE, Sprung CL, Douglas IS, Jaeschke R, Osborn TM, Nunnally ME, Townsend SR, Reinhart Thou, Kleinpell RM, Angus DC, Deutschman CS, Machado FR, Rubenfeld GD, Webb S, Beale RJ, Vincent J-Fifty, Moreno R. Surviving Sepsis Entrada: international guidelines for management of astringent sepsis and septic shock, 2012. Intensive Care Med. 2013;39:165–228. doi: 10.1007/s00134-012-2769-8. [PMC gratuitous commodity] [PubMed] [CrossRef] [Google Scholar]

4. Hock Ong ME, Lee Ng CH, Goh K, Liu N, Koh ZX, Shahidah N, Zhang TT, Fook-Chong Southward, Lin Z. Prediction of cardiac arrest in critically ill patients presenting to the emergency department using a automobile learning score incorporating eye rate variability compared with the modified early alarm score. Crit Care. 2012;16:R108. doi: 10.1186/cc11456. [PMC free commodity] [PubMed] [CrossRef] [Google Scholar]

5. Stevenson JE, Israelsson J, Nilsson GC, Petersson GI, Bath PA: Recording signs of deterioration in acute patients: The documentation of vital signs within electronic health records in patients who suffered in-hospital cardiac arrest. Wellness Information science J. 2016;22(one):21–33. doi:ten.1177/1460458214530136. Epub 2014 Apr 29. [PubMed]

vi. Gultepe East, Green JP, Nguyen H, Adams J, Albertson T, Tagkopoulos I. From vital signs to clinical outcomes for patients with sepsis: a automobile learning basis for a clinical decision support system. J Am Med Inform Assoc. 2014;21:315–25. doi: 10.1136/amiajnl-2013-001815. [PMC free article] [PubMed] [CrossRef] [Google Scholar]

7. Gardner-Thorpe J, Love N, Wrightson J, Walsh S, Keeling Due north. The value of Modified Early Warning Score (MEWS) in surgical in-patients: a prospective observational written report. Ann R Coll Surg Engl. 2006;88:571–5. doi: 10.1308/003588406X130615. [PMC complimentary article] [PubMed] [CrossRef] [Google Scholar]

9. Widgren BR, Jourak M. Medical Emergency Triage and Treatment Organisation (METTS): a new protocol in master triage and secondary priority decision in emergency medicine. J Emerg Med. 2011;40:623–8. doi: 10.1016/j.jemermed.2008.04.003. [PubMed] [CrossRef] [Google Scholar]

10. Pérez North, Nissen Fifty, Nielsen RF, Petersen P, Biering K. The predictive validity of RETTS-HEV as an acuity triage tool in the emergency department of a Danish Regional Hospital. Eur J Emerg Med. 2016;23:33–7. doi: 10.1097/MEJ.0000000000000173. [PubMed] [CrossRef] [Google Scholar]

11. Barfod C, Lauritzen MMP, Danker JK, Sölétormos G, Forberg JL, Berlac PA, Lippert F, Lundstrøm LH, Antonsen M, Lange KHW. Aberrant vital signs are strong predictors for intensive care unit admission and in-hospital mortality in adults triaged in the emergency department - a prospective accomplice written report. Scand J Trauma Resusc Emerg Med. 2012;20:28. doi: 10.1186/1757-7241-20-28. [PMC complimentary article] [PubMed] [CrossRef] [Google Scholar]

14. Sim I, Gorman P, Greenes RA, Haynes RB, Kaplan B, Lehmann H, Tang PC. Clinical decision back up systems for the practice of prove-based medicine. J Am Med Inform Assoc. 2001;8:527–34. doi: ten.1136/jamia.2001.0080527. [PMC free article] [PubMed] [CrossRef] [Google Scholar]

xv. Brilliant TJ, Wong A, Dhurjati R, Bristow E, Bastian L, Coeytaux RR, Samsa G, Hasselblad V, Williams JW, Musty MD, Fly 50, Kendrick AS, Sanders GD, Lobach D. Issue of clinical decision-support systems: a systematic review. Ann Intern Med. 2012;157:29–43. doi: 10.7326/0003-4819-157-1-201207030-00450. [PubMed] [CrossRef] [Google Scholar]

16. Rothman B, Leonard JC, Vigoda MM. Time to come of Electronic Health Records: Implications for Decision Back up. Mt Sinai J Med A J Transl Pers Med. 2012;79:757–768. doi: 10.1002/msj.21351. [PubMed] [CrossRef] [Google Scholar]

17. Elias P, Damle A, Casale M, Branson K, Churi C, Komatireddy R, Feramisco J. A Web-Based Tool for Patient Triage in Emergency Department Settings: Validation Using the Emergency Severity Index. JMIR Med informatics. 2015;3:e23. doi: 10.2196/medinform.3508. [PMC free article] [PubMed] [CrossRef] [Google Scholar]

18. Mandirola Brieux HF, Guillen S, La Rosa F, Moreno C, Benitez S. Nursing Software for Emergency Triage (NSET) Stud Wellness Technol Inform. 2015;216:942. [PubMed] [Google Scholar]

nineteen. Romero-Brufau S, Huddleston JM, Naessens JM, Johnson MG, Hickman J, Morlan BW, Jensen JB, Caples SM, Elmer JL, Schmidt JA, Morgenthaler TI, Santrach PJ. Widely used runway and trigger scores: are they ready for automation in do? Resuscitation. 2014;85:549–52. doi: 10.1016/j.resuscitation.2013.12.017. [PubMed] [CrossRef] [Google Scholar]

twenty. Liaw S-T. Clinical decision support systems: data quality direction and governance. Stud Health Technol Inform. 2013;193:362–9. [PubMed] [Google Scholar]

21. Juran JM, Gryna FM: Juran'southward Quality Command Handbook McGraw-Hilla - New York, 1988.

22. Kahn MG, Raebel MA, Glanz JM, Riedlinger M, Steiner JF. A pragmatic framework for single-site and multisite information quality assessment in electronic wellness record-based clinical inquiry. Med Care. 2012;fifty(Suppl):S21–ix. doi: ten.1097/MLR.0b013e318257dd67. [PMC costless article] [PubMed] [CrossRef] [Google Scholar]

23. Wang RW, Strong DM. Across Accurateness: What Data Quality Means to Data Consumers. J Manag Inf Syst. 1996;12:5. doi: 10.1080/07421222.1996.11518099. [CrossRef] [Google Scholar]

24. Weiskopf NG, Weng C. Methods and dimensions of electronic health tape data quality cess: enabling reuse for clinical research. J Am Med Inform Assoc. 2013;20:144–51. doi: 10.1136/amiajnl-2011-000681. [PMC free article] [PubMed] [CrossRef] [Google Scholar]

26. di Martino P, Leoli F, Cinotti F, Virga A, Gatta L, Kleefield S, Melandri R. Improving vital sign documentation at triage: an emergency department quality improvement project. J Patient Saf. 2011;7:26–9. doi: x.1097/PTS.0b013e31820c9895. [PubMed] [CrossRef] [Google Scholar]

27. Genes N, Chandra D, Ellis S, Baumlin K. Validating emergency department vital signs using a data quality engine for data warehouse. Open Med Inform J. 2013;7:34–nine. doi: 10.2174/1874431101307010034. [PMC costless commodity] [PubMed] [CrossRef] [Google Scholar]

28. Ward MJ, Froehle CM, Hart KW, Lindsell CJ. Operational data integrity during electronic health record implementation in the ED. Am J Emerg Med. 2013;31:1029–33. doi: x.1016/j.ajem.2013.03.027. [PubMed] [CrossRef] [Google Scholar]

29. Wager KA, Schaffner MJ, Foulois B, Swanson Kazley A, Parker C, Walo H. Comparison of the quality and timeliness of vital signs data using three different data-entry devices. Comput Inform Nurs. 2010;28:205–12. doi: 10.1097/NCN.0b013e3181e1df19. [PubMed] [CrossRef] [Google Scholar]

30. Vaismoradi M, Turunen H, Bondas T. Content analysis and thematic analysis: Implications for conducting a qualitative descriptive study. Nurs Wellness Sci. 2013;15:398–405. doi: x.1111/nhs.12048. [PubMed] [CrossRef] [Google Scholar]

31. Starks H, Trinidad SB. Choose your method: a comparison of phenomenology, soapbox analysis, and grounded theory. Qual Health Res. 2007;17:1372–80. doi: 10.1177/1049732307307031. [PubMed] [CrossRef] [Google Scholar]

32. Hsieh H-F, Shannon SE. Three approaches to qualitative content assay. Qual Health Res. 2005;15:1277–88. doi: ten.1177/1049732305276687. [PubMed] [CrossRef] [Google Scholar]

33. Kubicek H, Cimander R. Three dimensions of organizational interoperability. Insights from recent studies for improving interoperability frame-works. Eur J ePractice. 2009;6:ane–12. [Google Scholar]

34. Kaplan B, Harris-Salamone KD. Health Information technology success and failure: recommendations from literature and an AMIA workshop. J Am Med Inform Assoc. 2009;16:291–9. doi: 10.1197/jamia.M2997. [PMC costless commodity] [PubMed] [CrossRef] [Google Scholar]

35. Boonstra A, Broekhuis Thou. Barriers to the credence of electronic medical records by physicians from systematic review to taxonomy and interventions. BMC Health Serv Res. 2010;10:231. doi: ten.1186/1472-6963-x-231. [PMC free article] [PubMed] [CrossRef] [Google Scholar]

36. Miltner RS, Johnson KD, Deierhoi R. Exploring the frequency of blood pressure documentation in emergency departments. J Nurs Scholarsh. 2014;46:98–105. doi: x.1111/jnu.12060. [PubMed] [CrossRef] [Google Scholar]

37. Kohle-Ersher A, Chatterjee P, Osmanbeyoglu HU, Hochheiser H, Bartos C. Evaluating the barriers to point-of-care documentation for nursing staff. Comput Inform Nurs. 2012;30:126–33. doi: 10.1097/NCN.0b013e3182343f14. [PubMed] [CrossRef] [Google Scholar]

38. Carlson E, Catrambone C, Oder K, Nauseda Due south, Fogg 50, Garcia B, Dark-brown FM, Johnson ME, Johnson TJ, Llewellyn J. Point-of-intendance engineering science supports bedside documentation. J Nurs Adm. 2010;forty:360–5. doi: 10.1097/NNA.0b013e3181ee4248. [PubMed] [CrossRef] [Google Scholar]

39. Ratwani RM, Fairbanks RJ, Hettinger AZ, Benda NC. Electronic wellness record usability: analysis of the user-centered design processes of eleven electronic health record vendors. J Am Med Inform Assoc. 2015;22(6):1179–82. doi: 10.1093/jamia/ocv050. [PubMed] [CrossRef] [Google Scholar]

40. Ash JS, Sittig DF, McMullen CK, Wright A, Bunce A, Mohan V, Cohen DJ, Middleton B. Multiple perspectives on clinical determination support: a qualitative study of fifteen clinical and vendor organizations. BMC Med Inform Decis Mak. 2015;15:35. doi: 10.1186/s12911-015-0156-4. [PMC complimentary article] [PubMed] [CrossRef] [Google Scholar]

42. Marcos K, Maldonado JA, Martínez-Salvador B, Boscá D, Robles Thou. Interoperability of clinical determination-back up systems and electronic health records using archetypes: a case study in clinical trial eligibility. J Biomed Inform. 2013;46:676–89. doi: 10.1016/j.jbi.2013.05.004. [PubMed] [CrossRef] [Google Scholar]

43. Gøeg KR, Chen R, Højen AR, Elberg P. Content analysis of physical examination templates in electronic wellness records using SNOMED CT. Int J Med Inform. 2014;83:736–49. doi: 10.1016/j.ijmedinf.2014.06.006. [PubMed] [CrossRef] [Google Scholar]

44. Jian W-Southward, Hsu C-Y, Hao T-H, Wen H-C, Hsu M-H, Lee Y-L, Li Y-C, Chang P. Building a portable information and information interoperability infrastructure-framework for a standard Taiwan Electronic Medical Tape Template. Comput Methods Programs Biomed. 2007;88:102–xi. doi: 10.1016/j.cmpb.2007.07.014. [PubMed] [CrossRef] [Google Scholar]

45. Kopanitsa G, Veseli H, Yampolsky Five. Development, implementation and evaluation of an information model for archetype based user responsive medical information visualization. J Biomed Inform. 2015;55:196–205. doi: 10.1016/j.jbi.2015.04.009. [PubMed] [CrossRef] [Google Scholar]

46. Pettus D. Mobile medical device connectivity: real world solutions. Conf Proc IEEE Eng Med Biol Soc. 2004;5:3450–1. [PubMed] [Google Scholar]

---

Articles from BMC Medical Informatics and Decision Making are provided hither courtesy of BioMed Central

---

collinsselithe.blogspot.com

Source: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4893236/

Share this post