Background:

Public health calls to ensure equity in genomics and precision medicine necessitate a closer examination of how these efforts might differentially affect access to genetic services across demographic subgroups. This study set out to examine racial/ethnic disparities along the cancer genetic service delivery continuum.

Methods:

Retrospective data are drawn from 15 clinical sites across 6 U.S. States. Individuals who screened at-risk for hereditary cancer were: (i) referred/scheduled to see a genetic counselor (referral workflow), or (ii) offered genetic testing at the point-of-care (POC testing workflow). Logistic regression analyses evaluated the associations between race/ethnicity and several outcomes including appointment scheduling, genetic counseling, and genetic testing, controlling for demographics, clinical factors, and county-level covariates.

Results:

A total of 14,527 patients were identified at-risk. Genetic testing uptake was significantly higher at POC sites than referral sites (34% POC vs. 11% referral, P < 0.001). Race/ethnicity was significantly associated with testing uptake among all sites, with non-Hispanic Blacks having lower odds of testing compared with non-Hispanic Whites [aOR = 0.84; 95% confidence interval (CI), 0.71–1.00; P = 0.049]. Moreover, this disparity was observed at referral sites, but not POC sites. Among patients scheduled, non-Hispanic Blacks had lower odds of counseling (aOR = 0.28; 95% CI, 0.17–0.47; P < 0.001).

Conclusions:

Findings suggest that factors influencing genetic counseling show rates may be driving disparities in genetic testing.

Impact:

Strategies to reduce barriers to seeing a genetic counselor, including modifications to clinical workflow, may help mitigate racial/ethnic disparities in genetic testing.

This article is featured in Selected Articles from This Issue, p. 5

Genetic testing for hereditary breast and ovarian cancer (HBOC) and Lynch syndrome (LS) are classified as Tier 1 genomic applications by the Centers for Disease Control and Prevention (CDC; ref. 1). Under this framework, the CDC recommends genetic testing for HBOC and LS in accordance with available evidence-based guidelines and expert advice (1). However, fewer than one in five individuals meeting National Cancer Comprehensive Network (NCCN) eligibility criteria have undergone genetic testing (2). Of those eligible, over 70% of individuals have never discussed genetic testing with a health care provider (2). Many individuals are unaware that they are at risk (1) and fail to benefit from early detection of genetic variants that inform cancer prevention efforts (2).

Prior studies have found that barriers to testing disproportionally affect members of racial and ethnic minorities (3–5). In a population-based sample of women from Florida who had been diagnosed with invasive breast cancer at age 50 years or younger, discussion of genetic testing with a provider was 16 times less likely among Blacks compared with non-Hispanic Whites (3). Another study of 1,666 patients at a HBOC center found that non-Hispanic Blacks, Hispanics, and Asians were more likely to be referred to genetic services only after a cancer diagnosis and not based on family history, suggesting a missed opportunity for early detection of pathogenic variants and preventative cancer care (4). Lower rates of provider discussions and referrals ultimately result in lower uptake of genetic testing among racial and ethnic minority populations. A recent meta-analysis of 6,428 patients with ovarian cancer across 35 studies, found that only 26% of Black patients, and 14% of Asian patients completed genetic testing compared with 40% of White patients (5).

Public health calls to ensure equity in genomics and precision medicine necessitate a closer examination of how these efforts might differentially affect access to genetic services across demographic subgroups (6). Barriers to genetic testing influence disparities at all four levels of the health care system: (i) the individual patient, (ii) the care team, (iii) the organization, and (iv) the political and economic environment (7). The often complicated, multistage genetic screening and testing process contributes to the underuse of genetic testing; an issue that is exacerbated in underrepresented populations who have lower referral and testing rates from their providers (8). At the individual level, a lower awareness of testing among members of minority populations increases inequities in genetic testing (9). The care team may also reinforce disparities through implicit bias in referrals and inaccurate assumptions about the prevalence of hereditary cancer syndromes among certain racial and ethnic minority groups (10, 11). Political and economic barriers related to lack of insurance coverage and unclear costs of genetic testing widen the gap in testing uptake (11). Factors spanning multiple levels of the healthcare system such as cost, language barriers, and patient mistrust in the medical system (especially among African Americans) further exacerbate genetic testing disparities (11).

Interventions to improve the delivery of cancer genetic services have been examined across the continuum of genetic services delivery; they have included the identification of eligible patients, referral to genetic services, delivery of genetic counseling, coordination, and execution of genetic testing, and delivery of genetic results (12). Yet, studies examining racial/ethnic disparities often focus on a single outcome such as genetic testing uptake (4, 13, 14). Because there are multiple upstream processes along the continuum of care, the focus on a single outcome may limit our understanding of the drivers of reported disparities.

Our prior work demonstrated the differential impact of clinical workflow (e.g., traditional referral vs. point of care testing) on genetic testing rates, following the deployment of a digital system to facilitate the identification of patients at risk for hereditary cancer (15). The current study set out to identify and describe racial/ethnic disparities along the cancer genetic service delivery continuum. In particular, we focused on predictors of outcomes along each step in the delivery of genetic services and assessed whether there were differential outcomes by racial/ethnic subgroups at distinct points along the continuum of care. The identification of inequities at every stage of the genetic service delivery process will enhance our understanding of possible drivers of disparities along the care continuum and facilitate intervention efforts to ensure equitable access to genetic services.

Digital cancer genetic risk assessment program

Individuals complete a screening questionnaire through CancerIQ, a digital precision prevention platform that determines hereditary cancer risk based on the latest evidence-based guidelines (NCCN, Tyrer-Cuzick). The platform facilitates access to genetic services based on test results, helping providers use genetic information to predict, pre-empt, and prevent cancer across populations.

Study procedures

For the purposes of this study, data was made available from 15 health care sites (7 referral/scheduling, 8 POC) across 6 states for the time period from January 2019 to December 2019 (see Supplementary Table S1 for site characteristics). We selected clinical sites that had collected age, sex, personal cancer history, and race/ethnicity. Participants from the selected sites were excluded if they had missing race/ethnicity data.

Patients at each of the sites were screened via CancerIQ, either by completing questions on a tablet, or responding to questions asked by a staff member during the visit. On the basis of the clinical workflow deployed at the site, patients who screened at high risk were either: (i) referred/scheduled to see a genetic counselor in clinic (referral workflow) or (ii) offered genetic testing at the point-of-care (POC testing workflow). Within the referral workflow individuals screened at-risk were asked if they were interested in speaking with someone about their risk; those who indicated interest were called to schedule an appointment or had appointments scheduled on the day of screening. Within the POC testing workflow individuals screened at-risk were offered genetic testing, following a brief discussion about their risk. Actual testing (i.e., biospecimen collection) may have occurred either the same day or at a later date. Fig. 1 illustrates the clinical processes of each workflow.

Figure 1.

Clinical workflows. Figure outlines the process for patients within clinics deploying either a referral workflow or a point of care testing workflow.

Figure 1.

Clinical workflows. Figure outlines the process for patients within clinics deploying either a referral workflow or a point of care testing workflow.

Close modal

Outcome variables

We examined a primary outcome based on the uptake of genetic testing among high-risk patients, overall and stratified by workflow (referral and POC). In addition, within the referral workflow, the following two binary outcomes were examined: (i) whether a high-risk patient was scheduled for a genetic counseling appointment; and (ii) whether there was an uptake of genetic counseling among scheduled patients.

Predictors

We examined predictors of appointment scheduling, genetic counseling, and genetic testing at the patient level (age, personal cancer history, race/ethnicity), clinical level (workflow) and county level (Black, Hispanic, public insurance proportion of population). Age, sex, personal cancer history, race/ethnicity, and clinic workflow were collected through the CancerIQ digital platform. Because 93.11% of participants were female, we did not include sex in our adjusted model. Positive personal cancer history included HBOC-related cancers (Breast/Ovarian/Pancreatic/Prostate), LS-related cancers (CRC/Kidney/Gastric/Uterine), and other cancers. Race/ethnicity was categorized as non-Hispanic White, non-Hispanic Black, non-Hispanic Asian, any race Hispanic or Latino, and other (includes non-Hispanic Native Hawaiian or Pacific Islander, and non-Hispanic American Indian or Alaskan Native). The county-level proportions of residents identifying as Black or Hispanic, and those with public insurance, were obtained from the 2019 census data provided by the CDC for each respective site location. All values were scaled to represent a 1% point increase in the proportion of the population.

Statistical analysis

Descriptive statistics for continuous variables were presented as mean (SD) and categorical variables as frequencies (percentages). We compared patient characteristics by clinical workflows using two-sample t tests of means for continuous variables and Pearson χ2 tests for categorical variables.

We conducted logistic regression analysis using logit link function with binomial distribution to evaluate the associations between race/ethnicity and outcome variables (scheduling, counseling, and testing) controlling for demographics, clinical factors, and county-level covariates. First, we examined predictors of genetic testing overall, then stratified by the two workflows. Next, among the referral workflow, we examined predictors of scheduling genetic counseling appointments, and attending the genetic counseling appointment among those who were scheduled.

Because there were small cell counts for uptake status for some sites, and some sites lacking variable proportions of patients with scheduling, genetic counseling uptake, and genetic testing uptake, further adjustment of intra-class correlation (ICC) within sites resulted in unstable estimates or non-convergence of the models. Therefore, we conducted these logistic regression models without taking ICC into account. We used a two-sided type-1 error rate less than 0.05 for statistical significance. All analyses were conducted using SAS, version 9.4 (SAS Institute Inc).

Data availability

The data analyzed in this study are available upon request from the corresponding author.

Participant characteristics

In 2019, a total of 42,587 individuals with non-missing age, sex, personal cancer history, race, and ethnicity were screened across the 15 sites and 14,527 (34%) were identified as high risk and meeting NCCN genetic testing criteria for HBOC, LS, or both. Among those identified at high risk, 3,124 (22%) proceeded with genetic testing. Table 1 summarizes the characteristics of high-risk patients at all sites, and stratified by workflows. The majority of individuals at all sites were female (93%) and had no personal cancer history (78%). The total study population had the following racial/ethnic distribution: non-Hispanic White (81%), non-Hispanic Black (8%), non-Hispanic Asian (2%), any race Hispanic or Latino (8%), and other (1%). Among the 7,924 individuals at-risk at referral sites, 1,302 (16%) were scheduled for a genetic counseling appointment, 1,037 (13%) attended their genetic counseling appointment, and 910 (11%) completed genetic testing. Out of 6,603 at-risk patients at POC sites, 2,214 (34%) underwent genetic testing. Among those who completed POC testing, 1,634 (73.8%) underwent same day testing, while 580 (26.2%) were tested on a different day. POC patients had higher testing uptake than patients under a referral workflow (34% POC vs. 11% referral, P < 0.001; Table 1).

Table 1.

At-risk patient characteristics at clinical sites.

CharacteristicsAll Workflows N = 14,527Referral Workflow N = 7,924POC Workflow N = 6,603P value
Age in years, mean (SD) 56.17 (13.10) 58.15 (11.08) 53.80 (14.83) <0.0001 
Sex, n (%)    <0.0001 
 Male 1,001 (6.89%) 52 (0.66%) 949 (14.37%)  
 Female 13,526 (93.11%) 7,872 (99.34%) 5,654 (85.63%)  
Personal cancer history, n (%)    0.0903 
 No cancer history 11,399 (78.47%) 6,176 (77.94%) 5,223 (79.10%)  
 Positive cancer historya 3,128 (21.53%) 1,748 (22.06%) 1,380 (20.90%)  
Race/Ethnicity, n (%)    <0.0001 
 Non-Hispanic White 11,747 (80.86%) 6,663 (84.09%) 5,084 (77.00%)  
 Non-Hispanic Black 1,112 (7.65%) 737 (9.30%) 375 (5.68%)  
 Non-Hispanic Asian 283 (1.95%) 117 (1.48%) 166 (2.51%)  
 Any race Hispanic or Latino 1,184 (8.15%) 331 (4.18%) 853 (12.92%)  
 Otherb 201 (1.38%) 76 (0.96%) 125 (1.89%)  
Scheduled, n (%)  1,302 (16.43%)   
Counseled, n (%)  1,037 (13.09%)   
Tested, n (%) 3,124 (21.50%) 910 (11.48%) 2,214 (33.53%) <0.0001 
 Same-day testing   1,634 (73.80%)  
 Different-day testing   580 (26.20%)  
CharacteristicsAll Workflows N = 14,527Referral Workflow N = 7,924POC Workflow N = 6,603P value
Age in years, mean (SD) 56.17 (13.10) 58.15 (11.08) 53.80 (14.83) <0.0001 
Sex, n (%)    <0.0001 
 Male 1,001 (6.89%) 52 (0.66%) 949 (14.37%)  
 Female 13,526 (93.11%) 7,872 (99.34%) 5,654 (85.63%)  
Personal cancer history, n (%)    0.0903 
 No cancer history 11,399 (78.47%) 6,176 (77.94%) 5,223 (79.10%)  
 Positive cancer historya 3,128 (21.53%) 1,748 (22.06%) 1,380 (20.90%)  
Race/Ethnicity, n (%)    <0.0001 
 Non-Hispanic White 11,747 (80.86%) 6,663 (84.09%) 5,084 (77.00%)  
 Non-Hispanic Black 1,112 (7.65%) 737 (9.30%) 375 (5.68%)  
 Non-Hispanic Asian 283 (1.95%) 117 (1.48%) 166 (2.51%)  
 Any race Hispanic or Latino 1,184 (8.15%) 331 (4.18%) 853 (12.92%)  
 Otherb 201 (1.38%) 76 (0.96%) 125 (1.89%)  
Scheduled, n (%)  1,302 (16.43%)   
Counseled, n (%)  1,037 (13.09%)   
Tested, n (%) 3,124 (21.50%) 910 (11.48%) 2,214 (33.53%) <0.0001 
 Same-day testing   1,634 (73.80%)  
 Different-day testing   580 (26.20%)  

aAffected HBOC (Breast/Ovarian/Pancreatic/Prostate), affected Lynch (CRC/Kidney/Gastric/Uterine), affected other.

bIncludes non-Hispanic Native Hawaiian or Pacific Islander, and non-Hispanic American Indian or Alaskan Native.

Predictors of genetic testing as the primary outcome

Table 2 presents predictors of genetic testing among all high-risk individuals at referral and POC sites. In the adjusted multivariate logistic models, non-Hispanic Black patients had significantly lower odds of genetic testing compared with non-Hispanic White patients [aOR = 0.84; 95% confidence interval (CI), 0.71–1.00; P = 0.049]. Older age was associated with a decreased odds of testing (aOR = 0.98; 95% CI, 0.97–0.98; P < 0.001), while personal cancer history was associated with increased testing uptake (aOR = 1.16; 95% CI, 1.04–1.29; P = 0.006). The odds of testing were also statistically significantly lower among sites located in a county with a greater proportion of residents who were Black, Hispanic, and had public insurance. Individuals at POC clinical sites had substantially greater odds of genetic testing compared with those at referral sites (aOR = 3.09; 95% CI, 2.79–3.41; P < 0.001).

Table 2.

Predictors of genetic testing among individuals at high risk for hereditary cancers at all sites (N = 14,527).

NUnadjusted Bivariate Logistic Regression Models OR (95% CI)Adjusted Multivariate Logistic Regression Model OR (95% CI)
Patient level Age (in years) 14,527 0.98 (0.97–0.98), P < 0.001 0.98 (0.97–0.98), P < 0.001 
 Personal cancer history 
  No cancer history (ref) 11,399   
  Positive cancer historya 3,128 1.03 (0.94–1.13), P = 0.545 1.16 (1.04–1.29), P = 0.006 
 Race/Ethnicity 
  Non-Hispanic White (ref) 11,747   
  Non-Hispanic Black 1,112 0.80 (0.68–0.94), P = 0.006 0.84 (0.71–1.00), P = 0.049 
  Non-Hispanic Asian 283 1.68 (1.30–2.16), P < 0.001 1.03 (0.78–1.36), P = 0.838 
  Any race Hispanic or Latino 1,184 1.17 (1.01–1.34), P = 0.033 0.93 (0.79–1.10), P = 0.409 
  Otherb 201 1.69 (1.25–2.29), P = 0.001 1.18 (0.85–1.63), P = 0.321 
Clinic level Workflow 
  Referral (ref) 7,924   
  POC Testing 6,603 3.89 (3.57–4.24), P < 0.001 3.09 (2.79–3.41), P < 0.001 
County level Proportion of Population (per 1% point increase) 
  Black 14,527 0.99 (0.99–1.00), P = 0.053 0.99 (0.98–0.99), P < 0.001 
  Hispanic 14,527 1.01 (1.01–1.02), P < 0.001 0.99 (0.99–1.00), P < 0.001 
  Public Insurance 14,527 0.88 (0.87–0.89), P < 0.001 0.90 (0.90–0.91), P < 0.001 
NUnadjusted Bivariate Logistic Regression Models OR (95% CI)Adjusted Multivariate Logistic Regression Model OR (95% CI)
Patient level Age (in years) 14,527 0.98 (0.97–0.98), P < 0.001 0.98 (0.97–0.98), P < 0.001 
 Personal cancer history 
  No cancer history (ref) 11,399   
  Positive cancer historya 3,128 1.03 (0.94–1.13), P = 0.545 1.16 (1.04–1.29), P = 0.006 
 Race/Ethnicity 
  Non-Hispanic White (ref) 11,747   
  Non-Hispanic Black 1,112 0.80 (0.68–0.94), P = 0.006 0.84 (0.71–1.00), P = 0.049 
  Non-Hispanic Asian 283 1.68 (1.30–2.16), P < 0.001 1.03 (0.78–1.36), P = 0.838 
  Any race Hispanic or Latino 1,184 1.17 (1.01–1.34), P = 0.033 0.93 (0.79–1.10), P = 0.409 
  Otherb 201 1.69 (1.25–2.29), P = 0.001 1.18 (0.85–1.63), P = 0.321 
Clinic level Workflow 
  Referral (ref) 7,924   
  POC Testing 6,603 3.89 (3.57–4.24), P < 0.001 3.09 (2.79–3.41), P < 0.001 
County level Proportion of Population (per 1% point increase) 
  Black 14,527 0.99 (0.99–1.00), P = 0.053 0.99 (0.98–0.99), P < 0.001 
  Hispanic 14,527 1.01 (1.01–1.02), P < 0.001 0.99 (0.99–1.00), P < 0.001 
  Public Insurance 14,527 0.88 (0.87–0.89), P < 0.001 0.90 (0.90–0.91), P < 0.001 

Note: Statistically significant findings with a P-value less than 0.05 are indicated in bold.

aAffected HBOC (Breast/Ovarian/Pancreatic/Prostate), affected Lynch (CRC/Kidney/Gastric/Uterine), affected other.

bIncludes non-Hispanic Native Hawaiian or Pacific Islander, and non-Hispanic American Indian or Alaskan Native.

When analyses were stratified by workflows (Table 3), non-Hispanic Black patients had decreased odds of genetic testing within referral workflow sites (aOR = 0.60; 95% CI, 0.41–0.87; P = 0.008). Uptake of testing was 11% among non-Hispanic Whites compared with 8% among non-Hispanic Blacks at referral sites (P = 0.008; Fig. 2). However, among POC sites, non-Hispanic Black race/ethnicity was not associated with testing uptake (aOR = 1.07; 95% CI, 0.86–1.34; P = 0.521). The descriptive statistics showed that 37% of non-Hispanic Blacks completed genetic testing compared with 35% of non-Hispanic Whites at POC sites (P = 0.309). These patterns were similarly observed in the analysis of county-level data. Counties with a higher proportion of Black residents were associated with reduced testing uptake within the referral model (aOR = 0.90; 95% CI, 0.89–0.92; P < 0.001).

Table 3.

Predictors of genetic testing among individuals at high risk for hereditary cancers stratified by workflow.

Referral (N = 7,924)POC Testing (N = 6,603)
NUnadjusted Bivariate Logistic Regression Models OR (95% CI)Adjusted Multivariate Logistic Regression Model OR (95% CI)NUnadjusted Bivariate Logistic Regression Models OR (95% CI)Adjusted Multivariate Logistic Regression Model OR (95% CI)
Patient level Age (in years) 7,924 0.96 (0.96–0.97), P < 0.001 0.97 (0.96–0.98), P < 0.001 6,603 0.99 (0.99–0.99), P < 0.001 0.99 (0.99–0.99), P < 0.001 
 Personal cancer history 
  No cancer history (ref) 6,176   5,223   
  Positive cancer historya 1,748 1.43 (1.22–1.67), P < 0.001 1.15 (0.93–1.41), P = 0.197 1,380 0.88 (0.77–1.00), P = 0.042 0.96 (0.84–1.10), P = 0.526 
 Race/Ethnicity 
  Non-Hispanic White (ref) 6,663   5,084   
  Non-Hispanic Black 737 0.68 (0.52–0.91), P = 0.008 0.60 (0.41–0.87), P = 0.008 375 1.12 (0.90–1.39), P = 0.309 1.07 (0.86–1.34), P = 0.521 
  Non-Hispanic Asian 117 3.78 (2.54–5.62), P = 0.001 0.77 (0.44–1.36), P = 0.367 166 0.83 (0.60–1.16), P = 0.285 0.84 (0.60–1.18), P = 0.307 
  Any race Hispanic or Latino 331 1.77 (1.32–2.37), P < 0.001 1.01 (0.69–1.47), P = 0.973 853 0.67 (0.57–0.79), P < 0.001 0.87 (0.73–1.05), P = 0.151 
  Otherb 76 5.32 (3.34–8.49), P < 0.001 2.50 (1.25–5.01), P = 0.010 125 0.67 (0.45–1.01), P = 0.054 0.74 (0.49–1.11), P = 0.148 
County level Proportion of population (per 1% point increase) 
  Black 7,924 0.79 (0.78–0.81), P < 0.001 0.90 (0.89–0.92), P < 0.001 6,603 1.01 (1.01–1.02), P < 0.001 1.01 (1.00–1.01), P = 0.017 
  Hispanic 7,924 1.05 (1.04–1.07), P < 0.001 1.02 (1.00–1.04), P = 0.111 6,603 0.98 (0.98–0.98), P < 0.001 0.98 (0.98–0.99), P < 0.001 
  Public insurance 7,924 0.67 (0.66–0.69), P < 0.001 0.70 (0.68–0.72), P < 0.001 6,603 0.98 (0.97–0.99), P < 0.001 0.98 (0.97–0.99), P < 0.001 
Referral (N = 7,924)POC Testing (N = 6,603)
NUnadjusted Bivariate Logistic Regression Models OR (95% CI)Adjusted Multivariate Logistic Regression Model OR (95% CI)NUnadjusted Bivariate Logistic Regression Models OR (95% CI)Adjusted Multivariate Logistic Regression Model OR (95% CI)
Patient level Age (in years) 7,924 0.96 (0.96–0.97), P < 0.001 0.97 (0.96–0.98), P < 0.001 6,603 0.99 (0.99–0.99), P < 0.001 0.99 (0.99–0.99), P < 0.001 
 Personal cancer history 
  No cancer history (ref) 6,176   5,223   
  Positive cancer historya 1,748 1.43 (1.22–1.67), P < 0.001 1.15 (0.93–1.41), P = 0.197 1,380 0.88 (0.77–1.00), P = 0.042 0.96 (0.84–1.10), P = 0.526 
 Race/Ethnicity 
  Non-Hispanic White (ref) 6,663   5,084   
  Non-Hispanic Black 737 0.68 (0.52–0.91), P = 0.008 0.60 (0.41–0.87), P = 0.008 375 1.12 (0.90–1.39), P = 0.309 1.07 (0.86–1.34), P = 0.521 
  Non-Hispanic Asian 117 3.78 (2.54–5.62), P = 0.001 0.77 (0.44–1.36), P = 0.367 166 0.83 (0.60–1.16), P = 0.285 0.84 (0.60–1.18), P = 0.307 
  Any race Hispanic or Latino 331 1.77 (1.32–2.37), P < 0.001 1.01 (0.69–1.47), P = 0.973 853 0.67 (0.57–0.79), P < 0.001 0.87 (0.73–1.05), P = 0.151 
  Otherb 76 5.32 (3.34–8.49), P < 0.001 2.50 (1.25–5.01), P = 0.010 125 0.67 (0.45–1.01), P = 0.054 0.74 (0.49–1.11), P = 0.148 
County level Proportion of population (per 1% point increase) 
  Black 7,924 0.79 (0.78–0.81), P < 0.001 0.90 (0.89–0.92), P < 0.001 6,603 1.01 (1.01–1.02), P < 0.001 1.01 (1.00–1.01), P = 0.017 
  Hispanic 7,924 1.05 (1.04–1.07), P < 0.001 1.02 (1.00–1.04), P = 0.111 6,603 0.98 (0.98–0.98), P < 0.001 0.98 (0.98–0.99), P < 0.001 
  Public insurance 7,924 0.67 (0.66–0.69), P < 0.001 0.70 (0.68–0.72), P < 0.001 6,603 0.98 (0.97–0.99), P < 0.001 0.98 (0.97–0.99), P < 0.001 

Note: Statistically significant findings with a P-value less than 0.05 are indicated in bold.

aAffected HBOC (Breast/Ovarian/Pancreatic/Prostate), affected Lynch (CRC/Kidney/Gastric/Uterine), affected other.

bIncludes non-Hispanic Native Hawaiian or Pacific Islander, and non-Hispanic American Indian or Alaskan Native.

Figure 2.

Uptake of genetic services among at-risk patients by race/ethnicity. Figure displays the uptake of appointment scheduling, genetic counseling and genetic testing by race/ethnicity for patients at referral sites. Genetic testing uptake by race/ethnicity at point of care sites is also presented.

Figure 2.

Uptake of genetic services among at-risk patients by race/ethnicity. Figure displays the uptake of appointment scheduling, genetic counseling and genetic testing by race/ethnicity for patients at referral sites. Genetic testing uptake by race/ethnicity at point of care sites is also presented.

Close modal

No association between race/ethnicity and genetic testing was observed among Hispanics or Latinos of any race at referral sites (aOR = 1.01; 95% CI, 0.69–1.47; P = 0.973) or POC sites (aOR = 0.87; 95% CI, 0.73–1.05; P = 0.151). In contrast to non-Hispanic Blacks, patients from counties with a larger Hispanic population had lower testing uptake among POC sites (aOR = 0.98; 95% CI, 0.98–0.99; P < 0.001) but not among referral sites (aOR = 1.02; 95% CI, 1.00–1.04; P = 0.111).

Predictors of scheduling and genetic counseling within referral workflow sites

Among referral sites, we performed a subgroup analysis to examine predictors of scheduling a genetic counseling appointment. We found that patient race/ethnicity was not associated with appointment scheduling in the adjusted multivariate model (Table 4). The proportion of non-Hispanic White and non-Hispanic Black individuals scheduled to see a genetic counselor was 15% among both racial/ethnic groups (Fig. 2). In addition, among those with an appointment scheduled at the referral sites, we examined predictors of genetic counseling attendance. Non-Hispanic Black patients were significantly less likely to attend their genetic counseling appointment compared with non-Hispanic White patients (aOR = 0.28; 95% CI, 0.17–0.47; P < 0.001). The actual attendance rates among those high-risk patients with scheduled genetic counseling appointments was 12% for non-Hispanic Whites and 9% for non-Hispanic Blacks (P = 0.008; Fig. 2). County level factors—namely the proportion of individuals who were Black, Hispanic and had public insurance in the geographic location of the clinical sites—were also significantly associated with genetic counseling attendance rates (Table 5).

Table 4.

Predictors of scheduling among individuals at high risk for hereditary cancers at referral workflow sites (N = 7,924).

NUnadjusted Bivariate Logistic Regression Models OR (95% CI)Adjusted Multivariate Logistic Regression Model OR (95% CI)
Patient level Age (in years) 7,924 0.96 (0.96–0.97), P < 0.0001 0.97 (0.96–0.97), P < 0.0001 
 Personal cancer history 
  No cancer history (ref) 6,176   
  Positive cancer historya 1,748 1.22 (1.06–1.40), P = 0.006 0.97 (0.81–1.16), P = 0.739 
 Race/Ethnicity 
  Non-Hispanic White (ref) 6,663   
  Non-Hispanic Black 737 0.94 (0.76–1.17), P = 0.580 0.78 (0.58–1.05), P = 0.107 
  Non-Hispanic Asian 117 4.24 (2.92–6.14), P < 0.0001 1.28 (0.73–2.24), P = 0.389 
  Any race Hispanic or Latino 331 1.78 (1.37–2.30), P < 0.0001 0.98 (0.71–1.37), P = 0.9140 
  Otherb 76 4.44 (2.81–7.01), P < 0.0001 2.17 (1.13–4.16), P = 0.020 
County level Proportion of population (per 1% point increase) 
  Black 7,924 0.85 (0.83–0.86), P < 0.0001 0.91 (0.90–0.93), P < 0.0001 
  Hispanic 7,924 1.06 (1.05–1.07), P < 0.0001 1.02 (1.00–1.03), P = 0.1150 
  Public insurance 7,924 0.69 (0.67–0.70), P < 0.0001 0.70 (0.68–0.72), P < 0.0001 
NUnadjusted Bivariate Logistic Regression Models OR (95% CI)Adjusted Multivariate Logistic Regression Model OR (95% CI)
Patient level Age (in years) 7,924 0.96 (0.96–0.97), P < 0.0001 0.97 (0.96–0.97), P < 0.0001 
 Personal cancer history 
  No cancer history (ref) 6,176   
  Positive cancer historya 1,748 1.22 (1.06–1.40), P = 0.006 0.97 (0.81–1.16), P = 0.739 
 Race/Ethnicity 
  Non-Hispanic White (ref) 6,663   
  Non-Hispanic Black 737 0.94 (0.76–1.17), P = 0.580 0.78 (0.58–1.05), P = 0.107 
  Non-Hispanic Asian 117 4.24 (2.92–6.14), P < 0.0001 1.28 (0.73–2.24), P = 0.389 
  Any race Hispanic or Latino 331 1.78 (1.37–2.30), P < 0.0001 0.98 (0.71–1.37), P = 0.9140 
  Otherb 76 4.44 (2.81–7.01), P < 0.0001 2.17 (1.13–4.16), P = 0.020 
County level Proportion of population (per 1% point increase) 
  Black 7,924 0.85 (0.83–0.86), P < 0.0001 0.91 (0.90–0.93), P < 0.0001 
  Hispanic 7,924 1.06 (1.05–1.07), P < 0.0001 1.02 (1.00–1.03), P = 0.1150 
  Public insurance 7,924 0.69 (0.67–0.70), P < 0.0001 0.70 (0.68–0.72), P < 0.0001 

Note: Statistically significant findings with a P-value less than 0.05 are indicated in bold.

aAffected HBOC (Breast/Ovarian/Pancreatic/Prostate), affected Lynch (CRC/Kidney/Gastric/Uterine), affected other.

bIncludes non-Hispanic Native Hawaiian or Pacific Islander, and non-Hispanic American Indian or Alaskan Native.

Table 5.

Predictors of genetic counseling among high-risk individuals at referral workflow sites scheduled for an appointment (N = 1,302).

NUnadjusted Bivariate Logistic Regression Models OR (95% CI)Adjusted Multivariate Logistic Regression Model OR (95% CI)
Patient Level Age (in years) 1,302 1.00 (0.99–1.01), P = 0.713 1.00 (0.98–1.01), P = 0.647 
 Personal cancer history 
  No cancer history (ref) 977   
  Positive cancer historya 325 1.55 (1.11–2.17), P = 0.011 1.13 (0.77–1.66), P = 0.527 
 Race/Ethnicity 
  Non-Hispanic White (ref) 1,028   
  Non-Hispanic Black 108 0.39 (0.26–0.60), P < 0.001 0.28 (0.17–0.47), P < 0.001 
  Non-Hispanic Asian 51 1.52 (0.67–3.42), P = 0.314 0.56 (0.23–1.36), P = 0.199 
  Any race Hispanic or Latino 81 1.06 (0.59–1.90), P = 0.837 0.67 (0.35–1.26), P = 0.210 
  Otherb 38 3.86 (0.92–16.26), P = 0.065 2.19 (0.49–9.86), P = 0.308 
County Level Proportion of population (per 1% point increase) 
  Black 1,302 0.88 (0.86–0.91), P < 0.001 0.93 (0.89–0.96), P < 0.001 
  Hispanic 1,302 1.05 (1.02–1.08), P = 0.003 1.11 (1.07–1.14), P < 0.001 
  Public insurance 1,302 0.83 (0.80–0.86), P < 0.001 0.85 (0.82–0.89), P < 0.001 
NUnadjusted Bivariate Logistic Regression Models OR (95% CI)Adjusted Multivariate Logistic Regression Model OR (95% CI)
Patient Level Age (in years) 1,302 1.00 (0.99–1.01), P = 0.713 1.00 (0.98–1.01), P = 0.647 
 Personal cancer history 
  No cancer history (ref) 977   
  Positive cancer historya 325 1.55 (1.11–2.17), P = 0.011 1.13 (0.77–1.66), P = 0.527 
 Race/Ethnicity 
  Non-Hispanic White (ref) 1,028   
  Non-Hispanic Black 108 0.39 (0.26–0.60), P < 0.001 0.28 (0.17–0.47), P < 0.001 
  Non-Hispanic Asian 51 1.52 (0.67–3.42), P = 0.314 0.56 (0.23–1.36), P = 0.199 
  Any race Hispanic or Latino 81 1.06 (0.59–1.90), P = 0.837 0.67 (0.35–1.26), P = 0.210 
  Otherb 38 3.86 (0.92–16.26), P = 0.065 2.19 (0.49–9.86), P = 0.308 
County Level Proportion of population (per 1% point increase) 
  Black 1,302 0.88 (0.86–0.91), P < 0.001 0.93 (0.89–0.96), P < 0.001 
  Hispanic 1,302 1.05 (1.02–1.08), P = 0.003 1.11 (1.07–1.14), P < 0.001 
  Public insurance 1,302 0.83 (0.80–0.86), P < 0.001 0.85 (0.82–0.89), P < 0.001 

Note: Statistically significant findings with a P-value less than 0.05 are indicated in bold.

aAffected HBOC (Breast/Ovarian/Pancreatic/Prostate), affected Lynch (CRC/Kidney/Gastric/Uterine), affected other.

bIncludes non-Hispanic Native Hawaiian or Pacific Islander, and non-Hispanic American Indian or Alaskan Native.

This large observational study sheds light on some of the individual and clinic-level factors that influence the uptake of genetic services, including genetic counseling and testing. In particular, we found clear differences in uptake of genetic testing by clinical workflow and race/ethnicity. Overall, genetic testing uptake was significantly higher at POC sites compared with referral sites. Among all sites, non-Hispanic Blacks had lower odds of testing compared with non-Hispanic Whites. Notably, stratification by workflow revealed racial disparities in genetic testing uptake among the referral sites but not the POC sites. Further analyses within referral sites did not find racial disparities in scheduling appointments, but did show racial disparities in uptake of genetic counseling among non-Hispanic Blacks.

Although we did not detect racial differences in appointment scheduling, other studies have shown that Black patients are less likely to be referred and subsequently scheduled for genetic counseling (3, 9, 16). Our findings, however, were consistent with results reported by Doe and colleagues (17), who found that after implementing a comprehensive multidisciplinary care program, White and Black patients were offered genetic appointments at similar rates. Yet, the study found that Blacks were less likely to attend their appointment than Whites. Additional studies are needed to determine whether the implementation of comprehensive solutions to systematically assist with the identification and referral of at-risk patients may mitigate the often seen racial disparities in referral.

The differential uptake of genetic counseling among Blacks that was observed may be related to insurance coverage and cost of genetic services (8, 17). In one study, non-Hispanic Whites were more likely to have private insurance than non-Hispanic Blacks (P < 0.001); yet only 4.1% of patients who completed testing were uninsured, while 74% had private insurance (4). In another study of 69 patients with breast cancer, analyses that initially showed racial differences for genetic counseling and testing outcomes were no longer statistically significant once insurance was controlled for, suggesting that insurance status may be explaining the observed racial disparities (16). The current study was unable to examine individual level insurance status, which may account for the racial difference observed in testing uptake. Future studies are needed to shed additional light on the differential association of uptake by race, accounting for insurance status, income, and other factors associated with socioeconomic conditions.

In addition, non-Hispanic Blacks may receive less support coordinating care and accessing genetic counseling (9, 18). In an intervention study of mostly non-Hispanic Black females, transportation to care was a significant barrier for patients on Medicaid (19). Other reasons for the observed disparities may include lower education and awareness of genetic counseling among non-Hispanic Blacks (9, 11), which may influence their interest and motivation to attend an appointment. Multiple studies have shown that Black women were eager to receive genetic testing once they were made aware of the indications and implications (20–22). Therefore, it is essential that patients are well informed at the time they are identified at-risk and offered genetic counseling.

Our findings suggest that POC testing may be better suited for improving testing uptake among non-Hispanic Blacks than a traditional referral-based workflow. Many barriers to attending a genetic counseling appointment are eliminated when patients are offered immediate, same day POC testing. For example, the implementation of on-site, guideline-based genetic testing for patients with prostate cancer, increased compliance with testing from 33.3% to 98.7% (23). However, POC testing does not always take place on the same day the patient is screened (24). In our study, approximately three-quarters of patients completed same-day testing. Studies have demonstrated that testing, when completed on the same day, yields substantially higher testing rates than when patients return on a different day for testing (24). As such, potential hurdles preventing same day testing, may reintroduce other barriers that may result in non-completion of testing.

Although POC testing yields higher testing rates, those rates are only approximating 35% to 37% overall, suggesting that there is much to be learned about how best to implement this workflow to yield a greater impact on testing rates (15, 25). For example, the manner in which testing is offered and discussed (i.e., format or amount of pretest education) may influence patient decision-making. Our work has previously reported a wide variability in uptake of genetic testing across POC testing sites (range 14%–53%; 15), highlighting the need for future work to closely examine these and other contextual factors that might contribute to optimized outcomes. Although the current study did not observe disparities among the POC sites, other studies have found that mainstreaming genetic service delivery, (i.e., where nongenetics health professionals discuss and order genetic testing) has not overcome disparities in testing uptake (25). Specifically, provider biases in the recommendation and offering of testing continued to be observed following the implementation of universal POC genetic testing (25). Multiple environmental, clinical, and technical factors introduce bias in genetic testing, leading to inequitable access to services (26). Ultimately, these data suggest that significant work is still needed to address some of these known barriers within the context of POC testing to effectively implement the most effective strategies for improving access to genetic testing (12).

Notably, POC testing, while yielding higher testing rates, may not lead to improving health outcomes. This study did not examine the follow-up care for patients who underwent genetic testing. As such, the extent to which these downstream outcomes might differ by service delivery model, and whether previously reported disparities in medical management (e.g., prophylactic surgery), would be similarly observed is unknown (3, 27). Evaluation of possible overtreatment following testing is also needed, to determine whether there are additional harms from the deployment of POC testing approaches (28). Thus, the downside of any strategy, including POC testing, should be examined to ensure that any harms are not overlooked in ongoing efforts to streamline care delivery.

An important strength of our study is its large size and diverse sample, including patients from 15 clinical sites across 6 geographically dispersed states. Our study is novel in its focus on predicting multiple outcomes along the care delivery continuum. In addition, we explored important predictors of genetic service uptake including race/ethnicity, clinical workflow, age, and personal cancer history. Additional efforts to examine disparities in access along other points of the care continuum (e.g., patients approached for risk assessment and screening, clinical management following testing) are very much warranted.

This study has several limitations that are important to note. First, the digital system deployed in this study only collected a small set of patient demographics factors, and lacked other important sociodemographic data such as education and insurance status, which could not be accounted for in the models. Moreover, other clinic-level factors (except workflow), including the proportion of patients approached and/or completed the cancer risk assessment, were not available in the dataset to be examined. As such, the detected associations were subject to residual confounding. Second, referral-based testing can take a long time to complete, and outcomes may not have been fully captured during the time interval specified for data collection. Third, the majority of the participants in the ‘Other” racial/ethnic category were those whose race/ethnicity was unknown, which undermines our ability to interpret the meaning of the significant findings for that group. Apart from the inclusion of 27 non-Hispanic Native Hawaiian or Pacific Islanders, and 33 non-Hispanic American Indians or Alaskan Natives, the race/ethnicity of the remaining 141 participants (69%) were unknown, and may include bi- and multiracial individuals. Finally, some clinical sites had a small number of patients who were scheduled or tested. Further adjustment of ICC with these small cell counts resulted in model non-convergence. Therefore, the lack of ICC adjustment may have affected the statistical significance of the findings.

In summary, our study found that uptake of genetic testing differs across race/ethnicity and clinical workflow. Among referral workflow sites, non-Hispanic Black individuals were less likely to attend their genetic counseling appointment than non-Hispanic Whites, suggesting that barriers to seeing a genetic counselor may be driving disparities in genetic testing uptake. Findings from this study enhance the understanding of genetic testing disparities and aid the development of best practices to improve equitable access to genetic services.

H. Lu reports other support from CancerIQ outside the submitted work. No disclosures were reported by the other authors.

I.M. Wagner: Conceptualization, data curation, formal analysis, investigation, visualization, writing–original draft, writing–review and editing. Z. Xuan: Conceptualization, data curation, investigation, methodology, writing–review and editing. H. Lu: Resources, data curation, writing–review and editing. C. Wang: Conceptualization, supervision, investigation, writing-original draft, writing–review and editing.

The publication costs of this article were defrayed in part by the payment of publication fees. Therefore, and solely to indicate this fact, this article is hereby marked “advertisement” in accordance with 18 USC section 1734.

Note: Supplementary data for this article are available at Cancer Epidemiology, Biomarkers & Prevention Online (http://cebp.aacrjournals.org/).

1.
Centers for Disease Control and Prevention.
Tier 1 genomics applications and their importance to public health
.
Available from
: https://www.cdc.gov/genomics/implementation/toolkit/tier1.htm/.
2.
Childers
CP
,
Childers
KK
,
Maggard-Gibbons
M
,
Macinko
J
.
National estimates of genetic testing in women with a history of breast or ovarian cancer
.
J Clin Oncol
2017
;
35
:
3800
6
.
3.
Cragun
D
,
Weidner
A
,
Lewis
C
,
Bonner
D
,
Kim
J
,
Vadaparampil
ST
, et al
.
Racial disparities in BRCA testing and cancer risk management across a population-based sample of young breast cancer survivors
.
Cancer
2017
;
123
:
2497
505
.
4.
Chapman-Davis
E
,
Zhou
ZN
,
Fields
JC
,
Frey
MK
,
Jordan
B
,
Sapra
KJ
, et al
.
Racial and ethnic disparities in genetic testing at a hereditary breast and ovarian cancer center
.
J Gen Intern Med
2021
;
36
:
35
42
.
5.
Lin
J
,
Sharaf
RN
,
Saganty
R
,
Ahsan
D
,
Feit
J
,
Khoury
A
, et al
.
Achieving universal genetic assessment for women with ovarian cancer: are we there yet? A systematic review and meta-analysis
.
Gynecol Oncol
2021
;
162
:
506
16
.
6.
Khoury
MJ
,
Bowen
S
,
Dotson
WD
,
Drzymalla
E
,
Green
RF
,
Goldstein
R
, et al
.
Health equity in the implementation of genomics and precision medicine: a public health imperative
.
Genet Med
2022
;
24
:
1630
9
.
7.
National Academy of Engineering (US) and Institute of Medicine (U
S) Committee on Engineering and the Health Care System
,
Reid
PP
,
Compton
WD
,
Grossman
JH
, et al
.
A framework for a systems approach to health care delivery
.
National Academies Press (US)
.
2005
;
2
.
8.
Dusic
EJ
,
Theoryn
T
,
Wang
C
,
Swisher
EM
,
Bowen
DJ
, EDGE Study Team
.
Barriers, interventions, and recommendations: improving the genetic testing landscape
. Front Digit Health
2022
;
4
:
961128
.
9.
Reid
S
,
Cadiz
S
,
Pal
T
.
Disparities in genetic testing and care among black women with hereditary breast cancer
.
Curr Breast Cancer Rep
2020
;
12
:
125
31
.
10.
Ademuyiwa
FO
,
Salyer
P
,
Tao
Y
,
Luo
J
,
Hensing
WL
,
Afolalu
A
, et al
.
Genetic counseling and testing in African American patients with breast cancer: a nationwide survey of US breast oncologists
.
J Clin Oncol
2021
;
39
:
4020
8
.
11.
Frey
MK
,
Finch
A
,
Kulkarni
A
,
Akbari
MR
,
Chapman-Davis
E
.
Genetic testing for all: overcoming disparities in ovarian cancer genetic testing
.
Am Soc Clin Oncol Educ Book
2022
;
42
:
1
12
.
12.
Bednar
EM
,
Nitecki
R
,
Krause
KJ
,
Rauh-Hain
JA
.
Interventions to improve delivery of cancer genetics services in the United States: a scoping review
.
Genet Med
2022
;
24
:
1176
86
.
13.
Kurian
AW
,
Ward
KC
,
Howlader
N
,
Deapen
D
,
Hamilton
AS
,
Mariotto
A
, et al
.
Genetic testing and results in a population-based cohort of breast cancer patients and ovarian cancer patients
.
J Clin Oncol
2019
;
37
:
1305
15
.
14.
Dillon
J
,
Ademuyiwa
FO
,
Barrett
M
,
Moss
HA
,
Wignall
E
,
Menendez
C
, et al
.
Disparities in genetic testing for heritable solid-tumor malignancies
.
Surg Oncol Clin N Am
2022
;
31
:
109
26
.
15.
Wang
C
,
Lu
H
,
Bowen
DJ
,
Xuan
Z
.
Implementing digital systems to facilitate genetic testing for hereditary cancer syndromes: an observational study of 4 clinical workflows
.
Genet Med
2023
;
25
:
100802
.
16.
Choi
JJ
,
Fikre
T
,
Fischman
A
,
Buck
AK
,
Ko
NY
.
The role of race and insurance status in access to genetic counseling and testing among high-risk breast cancer patients
.
Oncologist
2022
;
27
:
832
8
.
17.
Doe
S
,
Petersen
S
,
Swain
M
.
Utilization of genetic testing in breast cancer treatment after implementation of comprehensive multidisciplinary care
.
Breast J
2020
;
26
:
911
6
.
18.
Mittendorf
KF
,
Knerr
S
,
Kauffman
TL
,
Lindberg
NM
,
Anderson
KP
,
Feigelson
HS
, et al
.
Systemic barriers to risk-reducing interventions for hereditary cancer syndromes: implications for health care inequities
.
JCO Precis Oncol
2021
;
5
:
1709
18
.
19.
Chaiyachati
KH
,
Hubbard
RA
,
Yeager
A
,
Mugo
B
,
Shea
JA
,
Rosin
R
, et al
.
Rideshare- based medical transportation for Medicaid patients and primary care show rates: a difference-in-difference analysis of a pilot program
.
J Gen Intern Med
2018
;
33
:
863
868
.
20.
Adams
I
,
Christopher
J
,
Williams
KP
,
Sheppard
VB
.
What black women know and want to know about counseling and testing for BRCA1/2
.
J Cancer Educ
2015
;
30
:
344
52
.
21.
Sheppard
VB
,
Graves
KD
,
Christopher
J
,
Hurtado-de-Mendoza
A
,
Talley
C
,
Williams
KP
.
African American women's limited knowledge and experiences with genetic counseling for hereditary breast cancer
.
J Genet Couns
2014
;
23
:
311
22
.
22.
Glenn
BA
,
Chawla
N
,
Bastani
R
.
Barriers to genetic testing for breast cancer risk among ethnic minority women: an exploratory study
.
Ethn Dis
2012
;
22
:
267
73
.
23.
Ramanathan
S
,
Korman
A
,
Korman
H
,
Balaraman
S
.
Positive impact of implementing a comprehensive genetic testing protocol for prostate cancer patients in a multidisciplinary uro-oncology practice. Letter
.
Urol Pract
2023
;
10
:
111
.
24.
DeFrancesco
MS
,
Waldman
RN
,
Pearlstone
MM
,
Karanik
D
,
Bernhisel
R
,
Logan
J
, et al
.
Hereditary cancer risk assessment and genetic testing in the community-practice setting
.
Obstet Gynecol
2018
;
132
:
1121
9
.
25.
Drogan
CM
,
Kindler
HL
,
Gao
G
,
Kupfer
SS
.
Outcomes of universal point-of-care genetic testing in diverse patients with pancreatic ductal adenocarcinoma
.
JCO Precis Oncol
2023
;
7
:
e2200196
.
26.
Matalon
DR
,
Zepeda-Mendoza
CJ
,
Aarabi
M
,
Brown
K
,
Fullerton
SM
,
Kaur
S
, et al
.
Clinical, technical, and environmental biases influencing equitable access to clinical genetics/genomics testing: a points to consider statement of the American College of Medical Genetics and Genomics (ACMG)
.
Genet Med
2023
;
100812
.
27.
Kim
Y
,
McCarthy
AM
,
Bristol
M
,
Armstrong
K
.
Disparities in contralateral prophylactic mastectomy use among women with early-stage breast cancer
.
NPJ Breast Cancer
2017
;
3
:
2
.
28.
Cragun
D
,
Weidner
A
,
Tezak
A
,
Clouse
K
,
Pal
T
.
Cancer risk management among female BRCA1/2, PALB2, CHEK2, and ATM carriers
.
Breast Cancer Res Treat
2020
;
182
:
421
8
.