Mahnoor Hafeez  ( Dow Institute of Radiology, Dow University of Health Sciences, Karachi, Pakistan. )
Amjad Sattar  ( Dow Institute of Radiology, Dow University of Health Sciences, Karachi, Pakistan )
Waqas Ahmed Farooqui  ( School of Public Health, Dow University of Health Sciences, Karachi, Pakistan. )

Objective: To determine whether there is inter-observer reliability between radiologists for reporting peritoneal carcinomatosis and computed tomography peritoneal carcinomatosis index estimation.

 

Method: The retrospective, cross-sectional study was conducted at Dow Institute of Radiology, Dow University of Health Sciences (DUHS), Ojha campus and comprised computed tomography scans done between December 1, 2019, to May 31, 2020, that were extracted from the institutional database searched using key words ‘peritoneal carcinomatosis’ and/or ‘serosal deposits’. The first readers had 1-4 years of post-fellowship experience, while the 2nd readers were senior radiologists. Inter-observer reliability was assessed quantitatively and qualitatively for 15 peritoneal sites using, among other tolls, the Sugarbaker computed tomography peritoneal carcinomatosis index. Data was analysed using SPSS 21.

 

Results: Out of 236 subjects with mean age 53.6±13.6 years, there were 173(73.3%) females and 63(26.7%) males. The most common primary cancer was ovarian 145(61.4%), followed by colon 26(11%). The size of peritoneal deposit was not reported in 75(31.8%) cases. Of the 15 sites analysed, good agreement was not found in 7(46.7%). There was an excellent intra-class correlation for measuring computed tomography peritoneal carcinomatosis index scores among the radiologists irrespective of the grade of the faculty (>0.90).

 

Conclusion: The inter-observer reliability was low, but good agreement for computed tomography peritoneal carcinomatosis index would encourage radiologists to use it in peritoneal cancer reporting.

 

Key Words: Computed tomography, Inter-observer reliability, Peritoneal carcinomatosis, Reporting, Agreement. (JPMA 73: 973; 2023) DOI: 10.47391/JPMA.6167

 

Submission completion date: 19-02-2022 — Acceptance date: 26-11-2022

 

Introduction

 

The peritoneum is a complex organ with multiple sub-sites, peritoneal ligaments and infoldings. Peritoneal carcinomatosis (PC) occurs secondary to dissemination of gynaecological and non-gynaecological neoplasm, including ovarian, endometrial, gastric oesophageal, colorectal, appendiceal, gallbladder and pancreatic malignancies. Computed tomography (CT), magnetic resonance imagining (MRI) and 18F fluorodeoxyglucose positron emission tomography (18F-FDG-PET) scans are non-invasive tools for peritoneal malignancy¹.

CT is a first-line investigation for the diagnosis and surveillance of PC because of its cost-effectiveness and widespread availability compared to the other modalities. Accurate estimation of PC can have a profound impact on the management of ovarian cancer patients. It can change the management plan from R0 to R1 resection. Deposits >2cm, implants at the diaphragm, lesser sac, porta hepatis, inter-segmental fissure, gallbladder fossa, or gastro-splenic or gastro-hepatic ligament also represent non-resectable disease². Akin O et al.³ studied CT for differentiating peri-hepatic metastases with and without liver parenchymal invasion (LPI) in patients with ovarian cancer via radiologists with different experience levels. They concluded that CT had sensitivity up to 100% and 80% for detecting peri-hepatic metastases with and without LPI respectively.

CT scanning creates a road map and surgical global positioning system (GPS) for PC. In terms of experience, clinicians’ feedback and literature search, smaller lesions are missed on CT⁴. It is a well-established phenomenon that double reading in diagnostic radiology can find discrepancies in the radiologists’ report, but it is time- and resource-consuming.

The current study was planned to determine inter-observer reliability (IOR) between radiologists for reporting PCs and CT PC index (CT-PCI) estimation. The null hypothesis was that IOR would be high.

 

Materials and Methods

 

The retrospective, cross-sectional study was conducted at Dow Institute of Radiology, Dow University of Health Sciences (DUHS), Ojha campus and comprised CT scans done between December 1, 2019, to May 31, 2020. After ethical approval from Institutional review board (IRB) of DUHS (Ref No. IRB/1771/ DUHS/Approval/2020), data was extracted from the institutional database using non-probability purposive sampling technique and searched using key words ‘peritoneal carcinomatosis’ and/or ‘serosal deposits’. The contrast-enhanced abdominal CT scans had been performed using 16-slice (Hitachi and Siemens) and 128-slice (GE Healthcare) scanners. The chosen slice thickness was 1mm (thin), reconstruction interval 0.5mm, volumetric acquisition, detector collimation 10mm, slice thickness pitch 1.75, number of detector channels 57 (16 rows), and the width of each detector channel 2mm.

All relevant CT reports of 15 sites were extracted. Images were acquired via thin-section (1.6mm) CT with overlapping section protocol with 16-slice and 64-slice scanners. Cases with incomplete information were excluded, and so were CT scans related to sites other than the 15 sites selected for the current study. Reporting had been done by Reader I individually, who was an academic radiologist working as Instructor, Senior Instructor or Assistant Professor with a different level of post-fellowship experience. The faculty was assigned a grade according to length of experience: Grade 1 = up to 2 years, Grade 2 = 2-5 years, Grade 3 = >4 years.

The retrospective analysis of all the scans was done by Reader II, an experienced consultant radiologist with pertinent background knowledge. The same CT scans were reviewed on digital imaging communication in medicine (DICOM) viewer with multi-planar reconstructions (MPRs) with appropriate windowing. All peritoneal sites were explored and hidden areas of the peritoneum at CT, including liver hilum, sub-phrenic area and the root of the mesentery were scrutinised (Figure-A). Reports were searched if size of the largest reachable peritoneal deposit for intervention was mentioned. The extent of lymphadenopathy was re-assessed.  Difference in PC terminology compared with international lexicon was noted. CT-PCI was calculated according to the number of peritoneal sites by two readers based on Sugarbaker´s CT-PCI score⁵, which is defined as the sum of sizes of the lesions in 13 abdomino-pelvic regions in clockwise direction (Figure B): S0 = no tumour visible; S1 = lesion size <0.5cm; S2 = lesion size 0.5-5cm, and S3 = lesion size >5cm. To avoid reporting bias, findings of both readers were recorded by separate data collectors, and data was kept anonymous.

 

 

Data was analysed using SPSS 21 and R software 4.0.3. For qualitative outcome (cancer sites), statistical agreement was checked between Readers I and II using observed agreement and AC1Gwetz statistic. For quantitative outcome (PCI score), intra-class correlation (ICC) was computed between the radiologists. IOR was categorised on the basis of ICC estimate with 95% confident interval (CI); <0.5 = poor, 0.5-0.75 = moderate, 0.75-0.9 = good, >0.9 = excellent. P<0.05 was considered statistically significant.

 

Results

 

Out of the 236 subjects with mean age 53.6±13.6 years, there were 173(73.3%) females and 63(26.7%) males. The most common primary cancer was ovarian 145(61.4%), followed by colon 26 (11%). The size of peritoneal deposit was not reported in 75(31.8%) cases. Supra-diaphragmatic lymphadenopathy was seen in 100(42.4%) cases, and was missed to be mentioned in 83(83%) of such cases. Among the first Readers, 23(9.7%) were instructors, 154(65.3%) were senior instructors and 58(24.6) were assistant professors (Table 1).

 

 

Of the 15 sites analysed, good agreement was not found in 7(46.7%). Senior instructors and assistant professors had a good agreement with the second reader, with agreement >70% and AC1Gwetz estimate >0.60 on 5(33.3%) sites; subphrenic space, intrahepatic fissure, porta-hepatis, splenic hilum, and lesser sac. On 2(13.3%) sites, fine peritoneal nodularity and subcapsular implant, a good agreement of the second reader with instructor was observed with agreement >70% and AC1Gwetz estimate >0.60 (Table 2).

 

 

There was an excellent ICC for measuring CT-PCI scores among the radiologists irrespective of the faculty grade (>0.90) (Table 3).

 

 

Discussion

 

Over the last decade, CT-PCI has emerged as an excellent tool for accurate PC estimation as it combines the distribution of peritoneal tumour with tumour size with results near analogous to laparotomy and laparoscopy⁵. Jónsdóttir B et al.⁶ found a cut-off value of CT-PCI >24 to achieve complete Cyto-reductive surgery (CRS). In the current study, mean CT severity index (CTSI) score ranged 28-30, with excellent reliability.

The prognostic significance of CT–PCI was evident from poster of Cecilia R et al. They also divided cavity into 13 regions and estimated maximum score up to 39 PCI of >20 as a relative contra-indication to intervention⁷. Low RN et al. in 2015⁸ studied accuracy of pre-operative CT and MRI in 22 patients retrospectively, who had undergone surgery, and concluded that CT had a bit lower sensitivity, (55%), specificity (86%) and accuracy (63%) than MRI; 95%, 70% and 88%, respectively.

Reporting by radiologists can have discrepancies that are inevitable^9,10. Geijer H. et al. in 2018 conducted a meta-analysis of 1,610 articles; 46 were then selected for analysis. They found that the discrepancy rate (DR) hovers around 22%, and concluded that double reading by a sub-specialist can lead to high rates of changed reports¹¹. Lauritzen P.M. et al.⁶ found a discrepancy rate of 14% during double reading of 1,071 consecutive abdominal CT examinations of surgical patients, retrospectively comparing the preliminary and final reports. De Bree E et al reviewed preoperative CT scans from 25 consecutive patients with PC from colorectal or appendiceal origin. The review was done by 2 blinded radiologists independently. They found a significant inter-observer differences⁷. Alternatively, studies^12,13 reported high agreement between radiologists when they reviewed the scans for multiple measurements on abdominal CT scans.

Sagrario LG et al. retrospectively evaluated 48 patients with ovarian carcinoma for PCI estimation, correlated findings with histopathology, and concluded that CT modality and PCI index were useful tools for treatment planning and prognosis¹⁴. A total of 49 cross-sectional cases were read by experienced and inexperienced radiologists for PCI assessment in a recent international study¹⁵.  Experienced radiologist was better in assessing both modalities) with higher agreement compared to inexperienced radiologist who was better at CT than MRI. In the current study, there was no significant variation in qualitative and quantitative IOR with experience and grade of faculty. For 8 PC sites, including omentum, Morrison’s pouch, serosal, mesentery, para-colic gutter, cul-de-sac, adnexa, umbilical’, poor IOR was seen (53%; 0.09).

Discrepancy and error is a truth and documented phenomena in Radiology^16,17. In this study, the size of peritoneal deposit and thickness of omentum had been taken into account if it was mentioned in reports for intervention. Enlarged cardiophrenic lymph nodes (LNs) aka supra-diaphragmatic nodes, predict worsening of survival in ovarian carcinoma patients¹⁸. In the current study, it was missed to be reported in 83 of the 100 cases. Significant differences in usage of PC terminology have been found. Omental deposits as 'beneath the anterior abdominal wall', Morrison’s pouch and sub-capsular as 'peri-hepatic', para-colic gutter as 'peri-colic', cul-de-sac and adnexa as 'pelvic' were the most frequently mis-reported terminology used in the study. Fultz PJ.¹⁹ documented that diagnostic aids, such as checklists and paired simultaneous readings, did not lead to an improved mean observer performance. This PC terminology flaw, detected on 1st reading is due to reasons including lack of appropriate feedback system, surgeon-radiologist communication gap, and lack of discrepancy meetings at the study site.

The study of CT-PCI and PC-IOR agreement were the strength of the current study, as, to the best of the authors’ knowledge, the matter has not been studied, and the index is neither very common among radiologists, nor is it practised in the country. It was an audit research to look into error rate of the department in this specific domain.

The current also has a few limitations. The CT findings were not compared with other imaging modalities, and patients were not followed up for the confirmation of per-operative findings.  Esquivel et al.²⁰ found that in 33% of subjects, pre-op CT-PCI score underestimated the extent of carcinomatosis. CT has poor sensitivity in determining small peritoneal tumours. On the other hand, MRI was not chosen for the study because of limited availability, bowel movement artifacts, claustrophobia and financial constraints. A checklist²¹ has been recently introduced for accurate reporting of PC at CT²¹, but was not applied in the current study.

 

Conclusion

 

The IOR was low, but there was good agreement for CT-PCI which would encourage radiologists to use it in PC reporting.

 

Disclaimer: The Abstract was presented orally at the 37th Annual Conference of Radiological Society of Pakistan held in Peshawar, Pakistan, in November 2021 and was presented at 6th International Colocon Conference in Karachi, Pakistan in December 2022.

 

Conflict of Interest: None.

 

Source of Funding: None.

 

References

 

1.      Patel CM, Sahdev A, Reznek RH. CT, MRI and PET imaging in peritoneal malignancy. Cancer Imaging. 2011; 11:123-39. doi: 10.1102/1470-7330.2011.0016.

2.      Nougaret S, Addley HC, Colombo PE, Fujii S, Al Sharif SS, Tirumani SH, et al. Ovarian carcinomatosis: how the radiologist can help plan the surgical approach. Radiographics. 2012; 32:1775-800. doi: 10.1148/rg.326125511.

3.      Akin O, Sala E, Moskowitz CS, Ishill N, Soslow RA, Chi DS, et al. Perihepatic metastases from ovarian cancer: sensitivity and specificity of CT for the detection of metastases with and those without liver parenchymal invasion. Radiology. 2008; 248:511-7. doi: 10.1148/radiol.2482070371.

4.      Pannu HK, Bristow RE, Montz FJ, Fishman EK. Multidetector CT of peritoneal carcinomatosis from ovarian cancer. Radiographics. 2003; 23:687-701. doi: 10.1148/rg.233025105.

5.      Lombardo Galera S, Espejo Herrera JJ, López Redondo C, Pérez Montilla ME, Muñoz Casares FC, Zurera Tendero LJ. Peritoneal Cancer Indexes in Ovarian Carcinomatosis: Correlation between CT and Intraoperative Results and Survival. Is the CT a Good Tool for Correlation? [Online] [Cite 2016 December 5]. Available from: URL: https://www.clinicsinsurgery.com/open-access/peritoneal-cancer-indexes-in-ovarian-carcinomatosis-correlation-between-ct-and-intraoperative-results-and-survival-is-the-ct-a-good-tool-for-correlation-2654.pdf

6.      Jónsdóttir B, Lomnytska M, Poromaa IS, Silins I, Stålberg K. The peritoneal cancer index is a strong predictor of incomplete cytoreductive surgery in ovarian cancer. Ann Surg Oncol. 2020; 28:244-51. doi: 10.1245/s10434-020-08649-6.

7.      Cecilia R, Muñoz CP, Nersesyan N, Pomares JJ, Carlos P, Palmero J. Key signs of peritoneal carcinomatosis and practical utility of Peritoneal Cancer Index in CT imaging. Eur Cong Radiol. 2018.

8.      Low RN, Barone RM, Lucero J. Comparison of MRI and CT for predicting the Peritoneal Cancer Index (PCI) preoperatively in patients being considered for cytoreductive surgical procedures. Ann Surg Oncol. 2015; 22:1708-15. doi: 10.1245/s10434-014-4041-7.

9.      Geijer H, Geijer M. Added value of double reading in diagnostic radiology, a systematic review. Insights Imaging. 2018; 9:287-301. doi: 10.1007/s13244-018-0599-0.

10.    Lauritzen PM, Andersen JG, Stokke MV, Tennstrand AL, Aamodt R, Heggelund T, et al. Radiologist-initiated double reading of abdominal CT: retrospective analysis of the clinical importance of changes to radiology reports. BMJ Qual Saf. 2016; 25:595-603. doi: 10.1136/bmjqs-2015-004536.

11.    De Bree E, Koops W, Kröger R, van Ruth S, Witkamp AJ, Zoetmulder FA. Peritoneal carcinomatosis from colorectal or appendiceal origin: correlation of preoperative CT with intraoperative findings and evaluation of interobserver agreement. J Surg Oncol. 2004; 86:64-73. doi: 10.1002/jso.20049.

12.    McErlean A, Panicek DM, Zabor EC, Moskowitz CS, Bitar R, Motzer RJ, et al. Intra-and interobserver variability in CT measurements in oncology. Radiology. 2013; 269:451-9. doi: 10.1148/radiology.13122665.

13.    Kembhavi PR, Chandramohan A, Kumara S, Palsetia D, Saklani A, Baheti AD, et al. Peritoneal Carcinomatosis Index (PCI) and advances in peritoneal disease management: What the radiologist should know. European Congress of Radiology-ECR 2019. doi.org/10.26044/ecr2019/C-2378

14.    Sagrario LG, Jose EH, Celia LR, Eugenia PM, Cristobal MC, Jesus ZT. Peritoneal Cancer Indexes in Ovarian Carcinomatosis: Correlation between CT and Intraoperative Results and Survival. Int J Clin Oncol Can Res. 2017; 2:75-8.

15.    Torkzad MR, Casta N, Bergman A, Ahlström H, Påhlman L, Mahteme H. Comparison between MRI and CT in prediction of peritoneal carcinomatosis index (PCI) in patients undergoing cytoreductive surgery in relation to the experience of the radiologist. J Surg Oncol. 2015; 111:746-51. doi: 10.1002/jso.23878.

16.    Brady AP. Error and discrepancy in radiology: inevitable or avoidable? Insights Imaging. 2017; 8:171-82. doi: 10.1007/s13244- 016-0534-1.

17.    Brady A, Laoide RÓ, McCarthy P, McDermott R. Discrepancy and error in radiology: concepts, causes and consequences.Ulster Med J. 2012; 81:3-9.

18.    Kolev V, Mironov S, Mironov O, Ishill N, Moskowitz CS, Gardner GJ et al. Prognostic significance of supradiaphragmatic lymphadenopathy identified on preoperative computed tomography scan in patients undergoing primary cytoreduction for advanced epithelial ovarian cancer. Int J Gynecol Cancer. 2010; 20:979-84. doi: 10.1111/IGC.0b013e3181e833f5.

19.    Fultz PJ, Jacobs CV, Hall WJ, Gottlieb R, Rubens D, Totterman SM et al. Ovarian cancer: comparison of observer performance for four methods of interpreting CT scans. Radiology. 1999; 212:401- 10. doi: 10.1148/radiology.212.2.r99au19401.

20.    Esquivel J, Chua TC, Stojadinovic A, Melero JT, Levine EA, Gutman M, et al. Accuracy and clinical relevance of computed tomography scan interpretation of peritoneal cancer index in colorectal cancer peritoneal carcinomatosis: a multi-institutional study. J Surg Oncol. 2010; 102: 565-70. doi: 10.1002/jso.21601.

21.    Chandramohan A, Thrower A, Smith SA, Shah N, Moran B. “PAUSE”: a method for communicating radiological extent of peritoneal malignancy. Clin Radiol. 2017; 72:972-80. doi: 10.1016/j.crad.2017.07.005