The prevalence of type 2 diabetes mellitus (T2DM) has increased worldwide, and diabetes itself is closely related to an increased risk of atherosclerotic cardiovascular diseases such as myocardial infarction and ischemic stroke, as well as mortality. As a result, population-based data have been widely used in epidemiologic studies [1-3] to identify individuals with diabetes and evaluate diabetes-related comorbidities and risk factors. The population-level classification of T2DM can also provide informative data to guide and prioritize populations at the greatest risk and those most likely to benefit from interventions and treatment. However, there is a limitation in the population-based claim database (DB) because accurate diagnoses cannot be made due to limited clinical and laboratory information, despite the advantage of the vast amount of data.

In Korea, two representative population-based DBs have been used, the Korea National Health and Nutrition Examination Survey (KNHANES) DB, with a cross-sectional design, and the National Health Insurance Service (NHIS) DB, with a national claims DB cohort design [4]. The Korean NHIS, a single-payer system for all residents, covers 97.1% of Koreans (approximately 50 million individuals), and this DB could be an efficient resource for diabetes research based on the entire population [5]. These big DBs have different advantages and disadvantages, depending on their characteristics.

Clinical measures, including glycosylated hemoglobin (HbA1c) and the oral glucose tolerance test (OGTT), are the gold standards for diagnosing diabetes [6]. However, it is difficult to routinely conduct an HbA1c test or OGTT in a study involving an entire population, especially for subjects with mild hyperglycemia. Instead, an operational definition was adopted to define diabetes using claims-based data and national health examination data in the NHIS DB. Generally, T2DM can be defined as the assignment of an International Classification of Disease, 10th Revision (ICD-10) code corresponding to T2DM (E11-14), with or without accompanying prescription codes for anti-diabetic drugs, or a high fasting glucose level (≥126 mg/dL) in the health check-up DB [7]. However, different operational definition criteria for diabetes were adopted for previous studies, depending on whether the diagnosis was based only on the corresponding ICD-10 codes [8,9], the use of concomitant drugs prescription were included [10-15], or fasting glucose results were included [16,17].

Whether the accuracy of defining diabetes based on claims data using diagnostic codes (ICD-10) with or without prescription codes (anti-diabetic drug use) is consistent with actual diabetes in the real-world is unknown. The quality of data must first be evaluated for fitness for use. Previous validation studies were performed based on comparisons with self-reports, telephone-based surveys, or medical chart reviews [18]. These methods may include biases, such as recall bias and selection bias, that affect accuracy and concordance. Our study aimed to evaluate the validity and reliability of the NHIS data-based definition of T2DM by comparing it with other population-based KNHANES data as a standard reference. The overall sensitivity, specificity, positive and negative predictive value, accuracy, and agreement were analyzed. We also compared the prevalence and concordance of T2DM when the two algorithms were applied, depending on whether the prescription codes and diagnostic codes were included in the criteria. To the best of our knowledge, this was the first study to validate the operational definition of T2DM using two big, linked Korean national DBs.

The Institutional Review Board of The Catholic University of Korea (IRB No.: VC18FESI0240) approved this study. The study was conducted in compliance with the Declaration of Helsinki. Written informed consent by the subjects was waived due to a retrospective nature of our study and anonymous and de-identified information was used for analysis.

Overall good validity and consistency of the diagnostic criteria using NHIS data were observed, which did not differ by age, sex, socioeconomic factors, or accompanied hypertension or dyslipidemia. When two diagnostic algorithms were applied to NHIS data according to whether the diagnostic codes were accompanied by prescription codes (algorithm 1) or not (algorithm 2), the prevalence of T2DM by algorithm 1 was lower than by algorithm 2, which was similar to the prevalence using the KNHANES data. In addition, although good reliability was observed for both algorithms, specificity and accuracy tended to increase in the algorithm that included both diagnostic and prescription codes (algorithm 1).

The prevalence of T2DM in the NHIS data using algorithm 1 (adopting both diagnostic and prescription codes) was lower, around 5.9% lower than when algorithm 2 (adopting only diagnostic codes) was applied. False positives (cases identified in NHIS claims data as having T2DM that were not diagnosed with T2DM by KNHANES criteria) increased when T2DM was defined only by diagnostic codes (8.7% in algorithm 2, 3.4% in algorithm 1). The overall prevalence of T2DM identified using algorithm 1 in this study was similar to the overall prevalence published in the 2021 Korea Diabetes Fact Sheet using KNHANES data (16.7%, approximately 6.05 million people) [15]. The mean HbA1c level in the NHIS+/KNHANES-group (false positives) was 5.9% using algorithm 1 and 5.7% using algorithm 2 in the study. There may be cases in which claims were issued for a T2DM diagnosis in subjects with prediabetes or early T2DM who did not require medications. Also, even though both algorithms (whether or not prescription claims data were included) provided good agreement based on the Kappa index, higher specificity, and accuracy for defining T2DM based on the NHIS were observed when claims for diagnostic codes were present along with prescription codes. When both diagnostic codes and prescription codes were included in the criteria for defining T2DM in the NHIS dataset, it helped to distinguish between patients who were in a prediabetic or early diabetic state and those who were in overt diabetes requiring treatment.

Concordance and the consistency of the diagnostic value based on NHIS criteria (algorithm 1) were not different according to age, sex, socioeconomic factors, and accompanied hypertension or dyslipidemia. The accuracy and specificity were over 90%, and the mean Kappa index showed good reliability (ranging from 0.68 to 0.78). These trends were consistent when algorithm 2 was applied (data not shown). A previous validation study compared accuracy and consistency using self-reports or telephone surveys as a reference standard [18]. Self-reports and telephone surveys are prone to recall bias, social desirability bias, poor understanding of the survey questions, incomplete knowledge, or their accurate diagnosis information. The literature review demonstrated that participants’ sociodemographic characteristics, such as age, gender, race, setting, and socioeconomic and health status, were associated with incomplete data linkage and the potential for systematic bias in reported outcomes [27]. Otherwise, our study used KNHANES data, a population-based surveillance system, as a reference standard. The KNHANES data has the advantage of minimizing selection bias compared to a diagnosis based on an electric medical chart review or interviews because the target population of the KNHANES comprises nationally representative non-institutionalized civilians in Korea. In addition, including clinical measures (HbA1c) as one of the diagnostic criteria in the reference standard for assessing validation can help overcome the potential limitation with systemic bias. Also, data linkage between the KNHANES and NHIS compensated for the shortcoming in the claims data, which was a lack of clinical information such as disease duration or glycemic control status, by adding information about self-reported surveys and urine or blood sample measurements in the KNHANES.

Validity of national claims administrative data was also evaluated in other countries such as Japan [28], Canada [29], and the USA [18]. Based on the Japanese national claim DB, the algorithm that contains both diagnosis-related codes for diabetes and medication codes had higher specificity (mean, 99.4% vs. 91.6%) and agreement (mean Kappa index, 0.80 vs. 0.49) than the algorithm that contains only diagnosis-related codes [28]. According to healthcare administrative data from Canada, compared with electronic medical records, the algorithm with the best specificity and PPV while maintaining sensitivity above 80% was either one hospitalization or physician claim and either one prescription for drug or diabetes-specific fee code at any time [29]. Validity of physician claims data-based on ICD-9 codes in the USA demonstrated that the sensitivity ranged from 26.9% to 97.0%, specificity ranged from 94.5% to 99.4%, and the Kappa index ranged from 0.8 to 0.9 [18]. Comparing the sensitivity, specificity, and Kappa agreement to other countries, the algorithm based on Korean NHIS data also demonstrated good validity and reliability.

Several limitations to this study should be considered. First, selection bias may have occurred because two-thirds of the subjects were excluded due to missing data on fasting glucose levels in the medical check-up DB or covariates in the KNHANES data, as well as cases where the person refused to provide personal information. In addition, only subjects aged 40 years or older were included in this study because national health check-up was conducted for 40 years or older. Second, among the diagnostic criteria in the KNHANES data used as a standard reference, questionnaires were also used to classify patients with T2DM through a self-reported survey. Other laboratory tests and data, such as the OGTT or hyperglycemia-accompanied symptoms, were not present in the data used to diagnose T2DM. As a result, the KNHANES data also did not fully reflect all patients with T2DM in real-world settings. Third, defining T2DM according to claims-based data can overlook patients with untreated diabetes or those who did not require treatment. Clinical factors such as disease duration, diabetes management status, or accompanied hypertension or dyslipidemia, were not assessed through the NHIS data. Despite these limitations, validating the operational diagnosis of T2DM by linking these two big national DBs, including clinical measures (HbA1c), represents a very important and timely investigation approach for future diabetes research in Korea.

In conclusion, population-based NHIS claims data can be useful in identifying subjects with T2DM by using diagnostic and prescription codes as diagnostic criteria in epidemiologic studies. The validity and accuracy of the population-based claims data for identifying T2DM were well documented and independent of sociodemographic and metabolic risk factors.