Abstract
Background
Patient reviews provide an important referral source for physicians and an opportunity to improve practice performance. This study's objective was to characterize the online reviews of hip and knee arthroplasty surgeons published by three of the industry's leading platforms.
Methods
A random sample of 1000 hip and knee arthroplasty surgeons across all 50 US states (10 hip and 10 knee surgeons per state) was generated using Google Search. A total of 7842 online reviews posted for those surgeons on Healthgrades, Vitals, and Google were analyzed. A range of surgeons, affiliated hospitals, and reviewer attributes was compared to identify significant predictors of patient satisfaction.
Results
The study cohort had 98.1% male surgeons with a mean age of 53.55 ± 8.94 years and mean experience of 26.43 ± 9.21 years. Younger age (p < 0.001), shorter years of experience (p < 0.001), and arthroplasty fellowship training (p < 0.001) were associated with more positive ratings. Reviewer anonymity, observed in 30.93% of all reviews, tended to correlate with more negative ratings (p < 0.001). Overall, 86.93% of patient remarks were positive, and only 74.81% of remarks centered on physician attributes. The five leading components of patient satisfaction were perceptions of physician competence (34.81%, p < 0.001), bedside manner (23.83%, p = 0.002), and communication (16.17%, p = 0.94); interactions with physician extenders (14.75%, p < 0.001); and wait time (2.73%, p < 0.001).
Conclusion
While most ratings of hip and knee arthroplasty surgeons were positive, more than a quarter of reviews were either not directly related to the individual surgeons or were submitted anonymously. Caution is advised regarding overreliance on patient experience surveys as predictors of physician performance.
Keywords
1. Introduction
In a value-based healthcare model, greater emphasis is placed on assessing the quality of care from the patient's perspective. in the United States, for example, the Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) and Press Ganey surveys have become widely adopted as quality metrics. Analogous to such surveys, but on a more personal level, online reviews directly assess patient experience with individual physicians. They are publicly accessible, making them readily available and more impactful when choosing providers. In a survey of a demographically representative sample of the US population, 60% of participants were at least aware of online patient reviews, and about 25% of them relied on such reviews when selecting a care provider.
1
Despite this trend, the role of online reviews in health care provision remains largely unclear, with attitudes varying greatly on their efficacy as a quality metric. Physicians, on one hand, may place substantially less trust in third-party online reviews. Patients, on the other hand, may value hearing about other patients' experiences.- Hanauer D.A.
- Zheng K.
- Singer D.C.
- Gebremariam A.
- Davis M.M.
Public awareness, perception, and use of online physician rating sites.
JAMA. 2014; 311https://doi.org/10.1001/jama.2013.283194
2
Still, aggregate data from multiple online rating sources have elucidated some prominent themes that significantly influence the patient review: staff empathy and demeanor, facility hygiene, and logistical burdens (e.g., waiting times).- Holliday A.M.
- Kachalia A.
- Meyer G.S.
- Sequist T.D.
Physician and patient views on public physician rating websites: a cross-sectional study.
J Gen Intern Med. 2017; 32https://doi.org/10.1007/s11606-017-3982-5
3
, - Smith R.J.
- Lipoff J.B.
Evaluation of dermatology practice online reviews.
JAMA Dermatol. 2016; 152https://doi.org/10.1001/jamadermatol.2015.3950
4
, - Hawkins J.B.
- Brownstein J.S.
- Tuli G.
- et al.
Measuring patient-perceived quality of care in US hospitals using Twitter.
BMJ Qual Saf. 2016; 25https://doi.org/10.1136/bmjqs-2015-004309
5
- Ranard B.L.
- Werner R.M.
- Antanavicius T.
- et al.
Yelp reviews of hospital care can supplement and inform traditional surveys of the patient experience of care.
Health Aff. 2016; 35https://doi.org/10.1377/hlthaff.2015.1030
Today, little is known about the content of online reviews of orthopaedic surgeons or the determinants of orthopaedic patient experience on e-commerce platforms. Prior studies were limited in scope to select databases, institutions, geographic regions, or orthopaedic surgery without delineation of different subspecialties.
6
, - Yu J.
- Samuel L.T.
- Yalçin S.
- Sultan A.A.
- Kamath A.F.
Patient-recorded physician ratings: what can we learn from 11,527 online reviews of orthopedic surgeons?.
J Arthroplasty. 2020; 35https://doi.org/10.1016/j.arth.2019.11.021
7
, 8
The objective of this national cross-sectional study was to provide a qualitative and quantitative analysis of online reviews specific to hip and knee arthroplasty surgeons. Specifically, we sought to: (1) understand what qualities—both physician- and non-physician-centered—were associated with patient experience, (2) characterize the review process used by the industry's leading platforms, and (3) explore whether ratings correlated with objective measures, such as complication rates of the surgeons' affiliated hospitals. As the influence of e-commerce platforms increases and higher emphasis is placed on optimizing the patient experience, a deeper understanding of online patient reviews of arthroplasty surgeons is essential.- Donnally C.J.
- McCormick J.R.
- Li D.J.
- et al.
How do physician demographics, training, social media usage, online presence, and wait times influence online physician review scores for spine surgeons?.
J Neurosurg Spine. 2019; 30https://doi.org/10.3171/2018.8.SPINE18553
2. Materials and methods
This study was classified as non-human subject research; therefore, it was exempt from institutional review board approval. Using Google Search, the first ten consecutive surgeons returned from the query “[hip or knee] replacement surgeon in [state]” were selected. In total, across all 50 US states, 500 hip and 500 knee arthroplasty surgeons were included in the study. Information related to each physician was extracted from the three leading online platforms for patient reviews: Google Reviews (Mountain View, CA), Healthgrades (Denver, CO), and Vitals (Lyndhurst, NJ).
9
Additional information was collected using Medicare's Compare Care and any associated personal or institutional websites. Compare Care is an online tool available through the Centers of Medicare and Medicaid Services that provides Medicare beneficiaries with a range of information about prospective providers and facilities. Of particular interest to this study, Compare Care was queried for 90-day complication rates post hip or knee arthroplasty associated with the hospitals where the study cohort practiced.- Ramkumar P.N.
- Navarro S.M.
- Chughtai M.
- La T.
- Fisch E.
- Mont M.A.
The patient experience: an analysis of orthopedic surgeon quality on physician-rating sites.
J Arthroplasty. 2017; 32https://doi.org/10.1016/j.arth.2017.03.053
10
Find healthcare providers: Compare care near you | Medicare.
https://www.medicare.gov/care-compare/
Date accessed: October 3, 2022
The present investigation excluded surgeons if they met any of the following criteria: (1) did not actively perform hip or knee arthroplasty in the US at the time of query; (2) had Google Search results that were paid or sponsored; (3) lacked patient reviews in the selected online platforms; or (4) could be not identified using Compare Care. Excluded surgeons were replaced with the next surgeon in the query list who met eligibility criteria. In total, across all 50 US states, 500 hip and 500 knee arthroplasty surgeons were included in the study.
Data on each surgeon's age, sex, years of experience, fellowship training (arthroplasty vs. other/none), hospital affiliation(s), and whether they had a personal website were collected. Hospital affiliations were identified through CMS Care Compare, which were then categorized as private or academic using the American Association of Medical Colleges' registry of academic hospitals. The complication rates for hip and knee arthroplasty for each surgeon's affiliated hospital were also collected. Finally, data on patient reviewers were captured, including their names (complete, partial, or anonymous) and the frequency of numeric-only ratings (vs. numeric and text comments).
When assessing patient comments, the first ten most recent ones for each surgeon published on Healthgrades were analyzed. Healthgrades was specifically chosen because the platform allowed reviewers to enter or withhold their names, had fewer word limits, and was the most comprehensive in terms of additional data collected. We felt this approach would result in the least biased and most comprehensive data compared to other platforms that mostly collected numeric or anonymous reviews. Patient comments were further divided into one or more remarks, which were categorized as either physician or non-physician centered. Physician-centered attributes included perceived competence, communication, and bedside manner. Non-physician attributes included comments about advanced practice providers, nurses, ancillary staff (e.g., billing, scheduling, etc.), and wait time. Ratings without text or those without clear explanations (e.g., an entry with only the word “good”) were grouped as numeric-only ratings.
A mean rating across the three platforms was calculated for all 1000 surgeons. Two groups scoring above and below this mean rating were generated and compared. Continuous data were summarized as means and standard deviations and compared using independent t-tests. Categorical variables were described as percentages and analyzed via two proportion z-tests. An α level of 0.05 was used for the determination of statistical significance.
3. Results
The study cohort was 98.10% male with a mean age of 53.55 ± 8.94 years and mean experience of 26.43 ± 9.21 years. Each surgeon was affiliated with 2.48 ± 1.32 hospitals on average. 53.30% of the arthroplasty surgeons had fellowship training in adult reconstructive surgery, 56.70% had affiliations with academic institutions, and 28.90% had personal websites. The mean rating across all three platforms was 4.42 ± 0.41 out of 5. Overall, Google ratings, which required at least a partial name of the reviewer, were significantly higher than those on Healthgrades or Vitals (p < 0.001). In contrast, reviews on Vitals, being mostly anonymous, were significantly lower than on the other two platforms (p < 0.001). 40.69% of Healthgrades reviews had a full reviewer name, 31.24% had a partial name, and 28.38% were anonymous.
After stratifying the review scores based on the mean, 598 profiles were in the above average group, and 402 profiles were in the below average group. Surgeons with above average ratings were younger (52.18 ± 9.16 years vs. 55.57 ± 8.21 years, p < 0.001), had fewer years of experience (25.25 ± 9.58 years vs. 28.19 ± 8.63 years, p < 0.001), and were fellowship-trained in adult reconstructive surgery (p < 0.001). A fewer number of hospital affiliations (2.54 ± 1.35 vs. 2.39 ± 1.28, p = 0.06) and lower 90-day complication rates at affiliated hospitals (2.34 ± 0.49% vs. 2.40 ± 0.50%, p = 0.053) were also found to be associated with above average reviews. Practice setting (academic vs. private) did not correlate with patient experience (p = 0.14). Table 1 summarizes the characteristics of the study cohort as well as the differences between above average and below average profiles.
Table 1Characteristics of the study cohort and predictors of patient experience.
| Variable | Total (N = 1000) | Above Average (N = 598) | Below Average (N = 402) | P Value |
|---|---|---|---|---|
| Age (years) | 53.55 ± 8.94 | 52.18 ± 9.16 | 55.57 ± 8.21 | <0.001 |
| Gender (male) | 98.10% | 98.66% | 97.26% | 0.18 |
| Surgical Experience (years) | 26.43 ± 9.21 | 25.25 ± 9.58 | 28.19 ± 8.63 | <0.001 |
| Number of Reviews | ||||
| 29.75 ± 39.35 | 37.22 ± 45.69 | 18.64 ± 23.34 | <0.001 | |
| Healthgrades | 50.59 ± 51.30 | 60.78 ± 59.50 | 35.43 ± 30.00 | <0.001 |
| Vitals | 45.37 ± 49.03 | 51.64 ± 58.04 | 36.05 ± 28.78 | <0.001 |
| Number of Hospital Affiliations | 2.48 ± 1.32 | 2.54 ± 1.35 | 2.39 ± 1.28 | 0.06 |
| CMS 90-Day Complication Rates at Affiliated Hospitals | 2.37 ± 0.50 | 2.34 ± 0.49 | 2.40 ± 0.50 | 0.053 |
| Arthroplasty Fellowship Training | 53.30% | 58.86% | 45.02% | <0.001 |
| Practice Setting | ||||
| Academic affiliation | 56.70% | 58.70% | 53.73% | 0.14 |
| Private only | 43.30% | 41.30% | 46.27% | 0.14 |
| Provider with Personal Websites | 28.90% | 30.77% | 26.12% | 0.13 |
| Reviewers | ||||
| Anonymous | 28.38% | 27.54% | 30.93% | <0.001 |
| Partial Name | 31.24% | 31.20% | 30.89% | 0.65 |
| Full Name | 40.69% | 41.26% | 38.18% | <0.001 |
In total, 15,218 remarks were analyzed. Of those, 13,230 (86.93%) were positive. In descending order, factors associated with patient experience were physician competence (34.81%, p < 0.001), bedside manner (23.83%, p = 0.002), communication (16.17%, p = 0.94), interaction with advanced practice providers (14.75%, p < 0.001), and wait time (2.73%, p < 0.001). Table 2 summarizes the determinants of patient experience through patient comments.
Table 2Determinants of patient experience through patient comments.
| Overall Remarks | Positive Remarks | Negative Remarks | P Value | |
|---|---|---|---|---|
| Total Remarks | 15,218 (100%) | 13,230 (86.93%) | 1988 (13.07%) | |
| Physician Competence | 34.81% | 35.39% | 30.99% | <0.001 |
| Physician Bedside Manner | 23.83% | 24.25% | 21.08% | 0.002 |
| Physician Communication | 16.17% | 16.15% | 16.25% | 0.94 |
| Interactions with Physician Extenders and Medical Assistants | 14.75% | 15.12% | 12.27% | <0.001 |
| Numeric-only Ratings | 5.47% | 5.86% | 2.92% | <0.001 |
| Wait Time | 2.72% | 1.75% | 9.21% | <0.001 |
| Interactions with Ancillary Staff | 2.24% | 1.48% | 7.29% | <0.001 |
4. Discussion
A few studies have investigated online patient reviews of orthopaedic surgeons in select geographic locations, but this is the first to explore the topic in a large nationally representative sample specific to hip and knee arthroplasty surgeons. We found patient reviews to be a multi-factorial reflection of both physician and non-physician factors. Younger age, fewer years of experience, and arthroplasty fellowship training were all significant predictors of positive patient experience. More than a quarter of patient remarks were not centered on physician attributes and were often associated with below average experience. There was also a tendency for higher patient ratings with hospital affiliations that had lower arthroplasty complication rates.
An interesting finding of this study was that more than a quarter of patient remarks were not attributed to surgeon-centered characteristics. This category—primarily consisting of interactions with physician extenders, ancillary staff, and wait times—appears to be consistent with previous studies. In a review of 11,527 online ratings of orthopaedic surgeons in a major metropolitan area, Yu et al.
6
noted a significant correlation with several non-physician-centered factors, such as ancillary staff friendliness. Likewise, in a single institution retrospective analysis, Mistry et al.- Yu J.
- Samuel L.T.
- Yalçin S.
- Sultan A.A.
- Kamath A.F.
Patient-recorded physician ratings: what can we learn from 11,527 online reviews of orthopedic surgeons?.
J Arthroplasty. 2020; 35https://doi.org/10.1016/j.arth.2019.11.021
11
found perception of staff friendliness among other non-surgeon factors to be intimately linked with positive patient experience. It should come as no surprise that patients, just like consumers of any other industry, value meaningful interactions and short wait times. This, however, may not be a simple task to accomplish with most practice models favoring high clinical volumes, which in turn may lead to longer wait times, shorter patient-physician interactions, and increased shunting of patient visits to advanced practice providers. This fact belies the dual challenge orthopaedic surgeons often face: the endeavor to provide a quality patient experience while fulfilling administrative and clinical volume expectations.- Mistry J.B.
- Chughtai M.
- Elmallah R.K.
- et al.
What influences how patients rate their hospital after total hip arthroplasty?.
J Arthroplasty. 2016; 31https://doi.org/10.1016/j.arth.2016.03.060
Another interesting finding of this study was that as reviews became more anonymous, ratings decreased. Anonymity may facilitate more open expression and transparency, but it may also increase the possibility of ranting and non-constructive criticism, thereby creating a missed opportunity to effectively engage with patients or assist practices in improving care. For example, when a patient provides a negative review of a provider due to issues such as parking, this penalizes the provider, who may have no bearing on the matter. Accordingly, the current state of patient reviews falls short of its purported mission to improve the patient experience. A significant proportion of these reviews represent a one-way outlet without room for providers or their staff to respond to negative comments or inquire further about personal experiences.
A third interesting finding of this study was the association between arthroplasty fellowship training and positive reviews. This finding may be explained by recent reports showing better patient outcomes in the hands of fellowship-trained orthopaedic surgeons. For example, in a single-institution analysis of fellowship-trained arthroplasty surgeons vs. their non-fellowship-trained colleagues, Mahure et al.
12
reported shorter surgical times, shorter hospitalization length, decreased opioid use, and higher rates of discharge to home among patients of fellowship-trained surgeons. In a separate multi-institutional retrospective review of 291 patients treated with a total hip arthroplasty for femoral neck fracture, Padilla et al.- Mahure S.A.
- Feng J.E.
- Schwarzkopf R.M.
- Long W.J.
The impact of arthroplasty fellowship training on total Joint arthroplasty: comparison of peri-operative metrics between fellowship-trained surgeons and non-fellowship-trained surgeons.
J Arthroplasty. 2020; 35https://doi.org/10.1016/j.arth.2020.05.027
13
found that fellowship training correlated with lower 90-day episode-of-care costs compared to those of non-fellowship-trained surgeons.- Padilla J.A.
- Gabor J.A.
- Ryan S.P.
- Long W.J.
- Seyler T.M.
- Schwarzkopf R.M.
Total hip arthroplasty for femoral neck fracture: the economic implications of orthopedic subspecialty training.
J Arthroplasty. 2020; 35https://doi.org/10.1016/j.arth.2020.01.047
There are a few limitations to this study. First, Google's search engine is based on a proprietary algorithm, so it is challenging to explain precisely how the top ten hip or knee surgeons were generated. Therefore, it is possible that if this search had been performed in a different location or with a different search engine, the query might have resulted in a different cohort of providers. Second, there was significant heterogeneity regarding content and rating methods among the various platforms. Third, given the public nature of online reviews and lack of established verification processes to prevent fake or paid submissions, such reviews if present were included in our analysis. Fourth, we indirectly measured the quality of care for the study cohort through the average rates of 90-day complications post hip or knee arthroplasty reported by Medicare's Compare Care for the surgeons' affiliated hospitals. Despite these limitations, our findings given a sample that is both nationally representative and robust in size may provide useful information to surgeons and researchers alike.
5. Conclusions
In conclusion, this is the first national cross-sectional analysis of online patient reviews specific to hip and knee arthroplasty surgeons. Despite the wide heterogeneity of online reviews, we found most to be positive. Younger age, arthroplasty fellowship training, and lower complication rates for affiliated hospitals were all significant predictors of positive ratings. Still, more than a quarter of the reviews were either not directly related to the individual surgeons or were submitted anonymously. As online reviews continue to play a greater role in clinical practice in an increasingly digital age with evolving patient demographics, caution is advised regarding overreliance on patient experience surveys as predictors of physician performance.
Declaration of competing interest
The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: Mohamad J. Halawi reports editorial/governing body memberships with The Journal of Bone and Joint Surgery, The Journal of Arthroplasty, and Arthroplasty Today. He also reports board membership/committee appointments with American Academy of Orthopaedic Surgeons and American Orthopaedic Association.
References
- Public awareness, perception, and use of online physician rating sites.JAMA. 2014; 311https://doi.org/10.1001/jama.2013.283194
- Physician and patient views on public physician rating websites: a cross-sectional study.J Gen Intern Med. 2017; 32https://doi.org/10.1007/s11606-017-3982-5
- Evaluation of dermatology practice online reviews.JAMA Dermatol. 2016; 152https://doi.org/10.1001/jamadermatol.2015.3950
- Measuring patient-perceived quality of care in US hospitals using Twitter.BMJ Qual Saf. 2016; 25https://doi.org/10.1136/bmjqs-2015-004309
- Yelp reviews of hospital care can supplement and inform traditional surveys of the patient experience of care.Health Aff. 2016; 35https://doi.org/10.1377/hlthaff.2015.1030
- Patient-recorded physician ratings: what can we learn from 11,527 online reviews of orthopedic surgeons?.J Arthroplasty. 2020; 35https://doi.org/10.1016/j.arth.2019.11.021
- Online ratings of orthopedic surgeons: analysis of 2185 reviews.Am J Orthoped. 2014; 43
- How do physician demographics, training, social media usage, online presence, and wait times influence online physician review scores for spine surgeons?.J Neurosurg Spine. 2019; 30https://doi.org/10.3171/2018.8.SPINE18553
- The patient experience: an analysis of orthopedic surgeon quality on physician-rating sites.J Arthroplasty. 2017; 32https://doi.org/10.1016/j.arth.2017.03.053
- Find healthcare providers: Compare care near you | Medicare.https://www.medicare.gov/care-compare/Date accessed: October 3, 2022
- What influences how patients rate their hospital after total hip arthroplasty?.J Arthroplasty. 2016; 31https://doi.org/10.1016/j.arth.2016.03.060
- The impact of arthroplasty fellowship training on total Joint arthroplasty: comparison of peri-operative metrics between fellowship-trained surgeons and non-fellowship-trained surgeons.J Arthroplasty. 2020; 35https://doi.org/10.1016/j.arth.2020.05.027
- Total hip arthroplasty for femoral neck fracture: the economic implications of orthopedic subspecialty training.J Arthroplasty. 2020; 35https://doi.org/10.1016/j.arth.2020.01.047
Article info
Publication history
Published online: March 06, 2023
Accepted:
March 4,
2023
Received in revised form:
January 13,
2023
Received:
October 8,
2022
Identification
Copyright
Published by Elsevier Inc.
