What solid evidence do systematic reviews provide about post-traumatic orbital reconstruction materials? An overview of systematic reviews

SYSTEMATIC REVIEWS

MAROLA, Luiz Henrique Godoi ^[1], TORRES, Luiz Henrique Soares ^[2], MOCHIZUKI JUNIOR, Ciro ^[3], MARINHO, Beatriz D’Aquino ^[4], CHIARELLI, Murillo ^[5], PEREIRA FILHO, Valfrido Antonio ^[6]

MAROLA, Luiz Henrique Godoi et al. What solid evidence do systematic reviews provide about post-traumatic orbital reconstruction materials? An overview of systematic reviews. Revista Científica Multidisciplinar Núcleo do Conhecimento. Year 09, Ed. 07, Vol. 01, pp. 47-81. July 2024. ISSN: 2448-0959, Acess link: https://www.nucleodoconhecimento.com.br/dentistry/post-traumatic-orbital, DOI: 10.32749/nucleodoconhecimento.com.br/dentistry/post-traumatic-orbital

ABSTRACT

Orbital fractures pose considerable challenges in the field of maxillofacial surgery. With advancements in materials engineering, various models and biomaterials have emerged for orbital reconstructions. Given the increasing number of systematic reviews (SRs) on orbital reconstructions, we aim to provide a comprehensive overview of SRs about biomaterials used in these procedures. Employing the PRIOR checklist, we scrutinized 14 SRs addressing materials for orbital reconstructions and their findings. The risk of bias was evaluated using the ROBIS tool, while the methodological quality of the reviews was assessed through the AMSTAR 2 tool. Our analysis revealed five low- or critically low-quality evidence, four of which had a strong recommendation for use and one that had a weak one. Despite the abundant literature on orbital reconstructions, high-quality evidence was notably absent. Still, this overview has generated pivotal and clear recommendations for surgical practice. We advocate for further randomized controlled trials featuring robust research designs to enhance the quality and reliability of evidence within this domain.

Key-words: Orbital Implants, Orbital Fractures, Absorbable Implants, Surgical mesh, Evidence-Based Practice.

1. INTRODUCTION

Orbital fractures represent significant challenges in maxillofacial surgery. Concerns such as the ideal time for the intervention, better surgical access, the use of customized implants, intraoperative navigation, and implant biomaterial are frequently debated issues.

As materials engineering advances, various models and biomaterials have been developed for orbital reconstructions. Evaluating these implants is crucial for clinical practice construction, and studies such as Randomized Controlled Trials (RCT), cohort studies, and Systematic Reviews (SR) play a critical role in this evidence-based process.

This practice should guide clinical decisions, and the critical analysis of SRs is a crucial step in this process. Every day, evidence-based practice gains more prominence, as it should be. Groups such as the Cochrane Collaboration¹, GRADE Collaboration^2–4, Joanna Briggs Institute⁵, NICE⁶, and others produce a considerable effort to create and constantly update tools related to the quality of evidence, with the understanding that it is not enough to have data; the data must be strongly reliable^7,8.

Given the increasing number of SRs on orbital reconstructions, we aim to provide a comprehensive overview of SRs about biomaterials used in these procedures. Through this analysis, we propose evaluating the evidence’s reliability and providing valuable insights for clinical practice.

2.MATERIALS AND METHODS

This SR overview was published on PROSPERO (CRD42023426823). We follow the PRIOR checklist, Preferred Reporting Items for Overview of Reviews⁹ (Appendix 1). The review question was: What solid evidence do systematic reviews provide about post-traumatic orbital reconstruction materials? The PECOS assessment is available in Table I.

2.1 SEARCH STRATEGY

The searches were conducted in May 2023 and repeated in March 2024. We used the databases PUBMED, COCHRANE, EMBASE, SCOPUS, WEB OF SCIENCE (via CAPES), LILACS (ENG, ESP, PT), Google Scholar (first 100 results), Science Direct, International Journal of Oral and Maxillofacial Surgery, and Journal of Oral and Maxillofacial Surgery. The complete search strategy descriptors can be found in Appendix 2. A manual search of the reference lists of the included articles was also conducted. Institutional repositories weren’t included as the study aims to evaluate evidence published in SRs.

The searches were conducted using a filter for the type of study “systematic reviews.” No other filters or restrictions were applied. The collected data was included in the EndNote platform for processing.

2.2 SCREENING PROCESS, DATA COLLECTION, AND OUTCOMES

Duplicate papers were identified and removed in EndNote; the remaining papers were added to the Rayyan platform¹⁰ to screen titles and abstracts and to apply eligibility criteria (Table I). Two independent and blinded authors (LHGM and LHST) performed this step, and a third author (MC) resolved the conflicts.

Two independent authors (LHGM and LHST) conducted the data collection, and it was reviewed by two other authors (CM and BDM) in a spreadsheet created for this purpose. A senior researcher (VAPF) supported the entire process.

We searched for results that presented safety and effectiveness in orbital reconstructions. Complications of interest were noted, including enophthalmos, diplopia, reduced ocular motility, and infection. No authors were contacted for unpublished data.

The primary studies used in the reviews have not been formally examined, which may result in overlap. Discrepant data found will be discussed in due course.

2.3 QUALITY OF REVIEWS AND BIAS ANALYSIS

LHGM and LHST conducted all steps of evaluating methodological quality and risk of bias, and the disagreements were resolved in consensus.

The systematic review quality was assessed using the AMSTAR 2¹¹. The final score can vary between high, moderate, low, and critically low quality. Bias analysis of the reviews was conducted using the ROBIS¹², which final bias score can be high, low, and uncertain.

The AMSTAR 2 and ROBIS inquire about the presence and clarity of various aspects of systematic reviews. When these aspects, methodological descriptions, results, or important caveats are absent or not clearly exposed in the text, the tendency is to classify them in that item as “high concern” or “low quality”.

To answer the AMSTAR 2 and ROBIS tools, we considered statistical syntheses of data (e.g., pooled outcome analysis and the like) as meta-analysis. Our understanding is that if an analytical sum or treatment were used to support a conclusion, it would need to comply with some principles present in the meta-analysis, such as weighting of results by sample size, study design, methodological quality, risk of bias, or similar. We did this to distinguish reviews that chose not to treat the data (usually due to heterogeneity) from those that did.

Study design classification, bias analysis, or methodological quality analysis of primary studies has not been conducted or reviewed. When the SRs carry out this evaluation, it will be made available with their results. This information will be found in the text using the abbreviation BQAPS (Bias and Quality Assessment of Primary Studies). We considered “not available” when the reviewers did not separate the primary works in any way; “not adequately” if the reviewers only classified the primary studies according to their study design (e.g., RCT) or did not provide the evaluation of each study individually; “adequately” if there was an individual evaluation and the score was made available. We do not analyze whether the method or tool chosen by the SRs is satisfactory/appropriate; we only analyze whether it has been made available to the reader adequately.

2.4 DATA SYNTHESIS, DATA CONFERENCE, AND CERTAINTY ASSESSMENT

No pooled or meta-analysis of the data was performed. The studies were approached separately and qualitatively assessed. If heterogeneity or sensitivity analysis is described in the SRs, they will be made available in due course.

We adopted the following for assessment:

1. We confronted the data, results, and discussion with the methodology used, and results BQAPS, ROBIS, and AMSTAR.
2. Authors CMJ, BDM, and MC reviewed the data for incorrect transcriptions, descriptive biases, and inconsistencies in descriptions.
3. These results were pooled and made available in a version of the GRADE tool that we adapted for quality of evidence and strength of recommendation.

3. RESULTS

3.1 SEARCH AND PATIENT CHARACTERISTICS

Figure I shows the PRISMA flowchart[13]. Overall, the studies evaluated more than 12,000 patients (with substantial overlap). Few of them described the patients’ sex (≥2:1 M/H). Age was described in a heterogeneous method (Table II). The list of articles read and not included, with their justifications, is in Appendix 3.

The general and statistical summaries of the reviews are in Table II and Table III, respectively.

3.2 QUALITY AND BIAS ANALYSIS AND GUIDELINES

The final grades for AMSTAR 2 and ROBIS are in Figures II and III ¹⁴, respectively.

Two reviews did not cite any tools to assess primary studies. Three (21.4%) were classified only according to the study design, one (7.1%) used Methodological items for non-randomized studies (MINORS), three New-Castle Ottawa Scale (NOS), and three used Cochrane tools. Twelve reviews (85.7%) followed the PRISMA checklist to describe the methodology. The PROSPERO registry was used in four (25.5%) reviews. Two reviews (14.2%) did not cite guidelines. Check Table II.

3.3 OUTCOMES

The observations BQAPS, AMSTAR 2, ROBIS, and notes in the “Enophthalmos” section should be extended by the reader in the other outcomes.

3.3.1 ENOPHTHALMOS

The results of Avashia et al. 2012¹⁵ were transcribed into Table III. BQAPS: not adequately. AMSTAR 2: critially low quality. ROBIS: high concern in bias.

The general data of Gunarajah and Samman 2013¹⁶ are available in Table III. Two primary studies were classified as quality level 1 (RCT); one compares Polydioxanone (PDS) (n=14) with Titanium mesh (n=14), and the postoperative enophthalmos was found in two patients in the PDS group; we found no mention of the statistical significance of this result. The other study compares PDS (n=12) with Collagen® (n=12); we did not find results on Collagen® in SR, and PDS did not present postoperative enophthalmos. BQAPS: not adequately. AMSTAR 2: critially low quality. ROBIS: low concern in bias.

Dubois et al. 2015¹⁷ suggested that combined floor and medial wall fractures have a higher chance of three-dimensional changes and enophthalmos, thus implying that patient-specific-implants (PSIs) may be more advantageous due to the accurate form. Four primary studies were classified as RCT, one of them multicentric*. One evaluated autologous auricular cartilage (n=8) vs. blade absorbable polyacid copolymer (n=12), another Collagen® membrane (n=12) vs. PDS foil 0.15 mm (n=12), another nasal autologous septal cartilage (n=11) vs. autologous conchal cartilage (n=11), and another perforated PDS foil 0.15 mm (n=14) vs. titanium dynamic mesh (n=15)*. Only the study that evaluated autologous nasal septal cartilage found a statistical difference in favor of this implant. BQAPS: not adequately. AMSTAR 2: critially low-quality. ROBIS: low concern in bias.

Wan et al. 2015¹⁸ found five studies that evaluated enophthalmos; of these, 3-27% of patients in the Computer-Assisted Technology (CAT) group – mirror image overlay, intraoperative navigation, and individualized preformed CAD/CAM implant – and 10-50% of the control group continued with enophthalmos at the last follow-up. A retrospective cohort with 6 stars in NOS (equivalent to moderate quality) found statistical significance in favor of CAT but did not provide data on exophthalmometry. The other four studies found no statistical differences. SR showed no advantage of CAT in the prevention of postoperative enophthalmos. BQAPS: adequately. AMSTAR 2: low-quality. ROBIS: low concern in bias.

Ramesh et al. 2018¹⁹ evaluated only absorbable implants. Two studies, level 2 of evidence by the Oxford Criteria, found enophthalmos: one with a case of enophthalmos after self-reinforced PGA (overall n=12), with 12/16; 14/16 bias by MINORS*; another (overall n= 16) with 37% enophthalmos after use of PDO, 9/16; 15/16 bias*. The reviewers point out the use of PDO: “When patients were stratified by defect size, fractures less than 1 cm² had adequate reconstruction.” They conclude that the absorbable implants show a late enophthalmos rate of 5% to 16% and that the PDO “may have slightly higher rates owing to rapid resorption in larger defects.” BQAPS: adequately. AMSTAR 2: critially low quality. ROBIS: high concern in bias.

*Note: The three papers on level 2 of evidence received different scores during the SR; we provide the two scores found.

Azzi et al. 2018²⁰ evaluated the use of orbital implants in a <16 years old population; the results are available in Table III. BQAPS: not available. AMSTAR 2: critially low quality. ROBIS: high concern in bias.

Oliver et al. 2020²¹ did not assess enophthalmos. BQAPS: not adequately. AMSTAR 2: critially low quality. ROBIS: high concern in bias.

Bourry et al. 2020²² found no statistical differences between the implants. The SUCRA ranking* has been transcribed into Table III. BQAPS: not available. AMSTAR 2: critially low quality. ROBIS: high concern in bias.

*Note: The SUCRA ranking should be interpreted cautiously since it has not been associated with the quality analysis of primary trials.

Maher et al. 2021²³ evaluated the use of PSIs. The authors found heterogeneities in the descriptions; most studies do not mention the values of enophthalmos, only the presence or absence of clinical enophthalmos (>2mm), with no apparent statistical differences between conventional techniques or those aided by PSI. Only one RCT report enophthalmos: the control group (n=5) was treated with conventional titanium mesh vs. the intervention group (n=5) with 3D printed models and manually molded titanium implants. The mean preoperative in the intervention group was 2.6 ± 0.8mm, which improved postoperatively to 0.35 ± 0.4mm. The mean preoperative in the control group was 3.8 ± 0.7mm, which improved to 2.4 ± 0.8mm postoperatively; the presence of statistical difference was not described in the review. Primary studies were not evaluated for bias. BQAPS: not adequately. AMSTAR 2: critially low quality. ROBIS: high concern in bias.

Hartmann et al. 2022²⁴ also evaluated the PSIs in orbital reconstructions. One study was described in detail, and the same is referenced in the summary of results of the review by Maher et al. 2021; thus, the groups, patients, means, and standard deviations are identical. In the Hartmann et al. 2022 review, this study was classified as a prospective clinical study, with an unclear risk of bias as assessed by the Cochrane tool. The reviewers emphasized favorable statistical significance for using PSIs in this study. BQAPS: adequately. AMSTAR 2: low quality. ROBIS: high concern in bias.

Kotecha et al. 2022²⁵ did not find statistical differences between surgeries performed with or without the aid of PSIs, with substantial heterogeneity (I²= 82.3%). Two primary studies found statistical differences in favor of PSI; one is the same as the one cited by Maher et al. 2021 and Hartmann et al. 2022. The SR of Kotecha et al. 2022²⁵ was rated as “some concerns” in bias analysis by Cochrane RoB2. The other is a retrospective cohort, with a moderate risk of bias by NOS. In this one, the conventional group (n=27) was operated with MEDPOR Titan ® vs. PSI group (n=29) operated with MEDPOR TITAN® – 3D printed. We did not find further data on the study. BQAPS: adequately. AMSTAR 2: critially low quality. ROBIS: low concern in bias.

Murray-Douglass et al. 2022²⁶ showed that some 3D aids were effective in reducing enophthalmos (Table III), with small heterogeneity (I²= 11.33%), and that the quality of the study* did not interfere with the effect size or heterogeneity. The reviewers conclude that 3D printing can help reduce enophthalmos but cannot quantify how much of this benefit is conferred on 3D alone. BQAPS: not adequately. AMSTAR 2: low quality. ROBIS: low concern in bias.

*Note: It is worth noting that the reviewers acknowledged using a “simple tool” for quality assessment due to the variability of the articles and that articles with high scores may not necessarily be of high quality when evaluated by other tools.

Singh et al. 2023²⁷ did not perform a meta-analysis of the data found on enophthalmos when comparing Manual Free-Hand-Shaped (MFS) vs 3D-Printed Model-Based (3DP). A retrospective study evaluating MFS (n=27) vs 3DP (n=29) showed statistical differences in favor of 3DP. The same RCT was described in the summary of results by Maher et al. 2021²³; here it was described with significance favorable to the 3DP. Another retrospective study comparing MFS (n=13) vs 3DP (n=17) found no differences between the groups. One RCT found significance favorable to 3DP (n=23) compared to MFS (n=16). All were classified as “high risk of bias” by RoB2. BQAPS: adequately. AMSTAR 2: critially low quality. ROBIS: low concern in bias.

The data described by Abukhder et al. 2024²⁸ were transcribed into Table III. All primary studies were classified as moderate quality by NOS. BQAPS: adequately. AMSTAR 2: critially low quality. ROBIS: low concern in bias.

3.3.2 DIPLOPIA

The data from Avashia et al. 2012¹⁵, Gunarajah and Samman 2013¹⁶, Azzi et al. 2018²⁰, and Abukhder et al. 2024²⁸ were transcribed in Table III. The SR of Gunarajah and Samman 2013¹⁶ showed no pre- or postoperative diplopia in the level 1 studies. Bourry et al. 2020²² and Kotecha et al. 2022²⁵ found no statistical differences in their meta-analyses. The SUCRA score ²² is made available in Table III.

Dubois et al. 2015¹⁷, of the four studies classified as RCT, two reported postoperative diplopia. One compared nasal septal cartilage (n=11) vs. conchal cartilage (n=11) and obtained 9% post-op diplopia in both groups; preoperative data are unavailable. The other study evaluated perforated PDS foil 0.15 mm (n=14) vs. titanium dynamic mesh (n=15); the preoperative diplopia rates were 75% and 88%, respectively, and postoperative diplopia was 50% for both. Statistical significance data for these results are not available.

Wan et al. 2015¹⁸ found four studies that reported postoperative diplopia. Three showed significant improvement in diplopia in the CAT vs. control groups: 51 vs. 60% (historically control trial with 8 stars in NOS, equivalent to high quality), 2 vs. 10% (prospective cohort study with 9 stars in NOS), and 17 vs. 88% (retrospective cohort study with 6 stars in NOS). A prospective cohort study with 7 stars in NOS found no significant difference. The reviewers conclude by suggesting an advantage of the use of CAT for the resolution of diplopia.

Ramesh et al. 2018¹⁹ found a paper level 2 of evidence and risk of bias 9/16; 15/16 who showed postoperative diplopia due to significant hypoglobus and enophthalmos after PDS reconstruction.

The SR of Maher et al. 2021²³ described a multicenter controlled trial: the Control Group – CG (n=84) –  was reconstructed with titanium mesh, and the Intervention Group – IC (n=61), received manually molded titanium implants in 3D printed models. The CG presented 65% of preoperative and 25% postoperative diplopia, and the IC presented 39% and 25%, respectively. This difference was not statistically significant.

Murray-Douglass et al. 2022²⁶ observed that, individually, contour model, mould, and surgical planning effectively reduce diplopia. In the general evaluation, 3D printing was also effective, with small heterogeneity (I²= 10.91%), and the quality of the study did not interfere with the effect size or heterogeneity. Although they found that 3D printing helped reduce diplopia, they could not quantify how much of this benefit is conferred on 3D alone.

Singh et al. 2023²⁷ did not perform a meta-analysis of data on diplopia in the MFS vs. 3DP groups. Primary studies – composed of retrospective (n=13 vs. 17), prospective controlled multicenter trial (n=95 vs. 100), and RCT (n=16 vs. 23) – showed, for the most part, no statistical differences favorable to the 3DP group. A retrospective study (n=12 vs. 12) showed a difference in favor of the 3DP group. All works in this SR are at high risk of bias by RoB2.

3.3.3 FRACTURE PATTERNS, SIZE, OR ORBITAL VOLUME

Gunarajah and Samman 2013¹⁶ provided an infographic indicating which materials would be indicated considering the size of the defect. We suggest reading it in the original paper.

Ramesh et al. 2018¹⁹ suggested that absorbable implants are indicated for isolated fractures of the medial wall and floor where the bony buttresses are intact, and the implants should serve as a barrier rather than load-bearing. They also suggest that PDO implants, pure PDLLA (faster resorption time), and PLLA (increased risk of delayed inflammation) are not indicated for orbital reconstructions.

Bourry et al. 2020²² provided an infographic indicating for Jaquiéry type 1 fracture: PL(DL/LA), PDS, and Polyglactin/PDS; Jaquiéry 2: PL(DL/LA); Jaquiéry 3, 4 and 5: Titanium, and Porous polyethylene.

In the SR of Maher et al. 2021²³, five papers reported the complete orbital volume data and were divided into three groups: Group 1(G1) – manual molding PSI on the 3D printed model; Group 2 (G2): fully individualized and manufactured from the 3D printer; and Group 3 (G3): fabrication of template from 3D printer. G1 has two prospective cohorts (n=12 and 104) and one retrospective cohort (n=104), G2 has one retrospective cohort (n=15), and G3 has one prospective cohort (n=11). In G1, the three studies analyzed observed a statistically significant improvement in the mean volume of the fractured orbit comparing the preoperative and postoperative CT scans, and the volume of the affected orbit after the intervention was similar to the volume of the unaffected orbit. In G2, a statistically significant improvement in volume was observed compared to the preoperative period, and the contralateral orbit presented a significantly larger volume than the operated one. G3 showed a statistically significant improvement compared to the preoperative moment and did not differ from the contralateral orbit.

Hartmann et al. 2022²⁴ cited two studies (a multicenter prospective controlled trial and retrospective case series) in which PSI was shown to be more accurate regarding orbital volume. We did not find data on the significance of this difference.

During the meta-analysis, Kotecha et al. 2022²⁵ found no statistical differences between the PSI and conventional groups. Five papers evaluated orbital volume, and only three provided volumetric data amenable to pooling in meta-analysis; they will be marked with “§. ” Two studies did not find statistical differences between PSIs and conventional: a prospective cohort§ with a low risk of bias in NOS and a retrospective cohort§ with a moderate risk of bias. Three studies found statistical differences in favor of PSIs: A prospective cohort with a low risk of bias, a retrospective cohort with a low risk of bias, and a retrospective cohort§ with a moderate risk of bias. The PSI reconstructions were mainly performed with Titanium.

Murray-Douglass et al. (2022)²⁶ found no statistical differences when evaluating orbital volume between operated and non-operated orbits in favor of any 3D aids researched.

Singh et al. 2023²⁷ cited a multicenter trial supported by AO Foundation – CMF that compared MFS (n=95) vs. 3DP (n=100), where 3DP had statistically significantly higher accuracy compared to the MFS group. In a retrospective study (n=12 vs. 10), the authors compared the postoperative orbital volume with the non-traumatized orbit; the MFS group showed a significant difference in volume compared to the non-traumatized orbit, while the 3DP group did not show this difference. In another retrospective study (n=38 vs. 44), the authors found a statistically larger residual defect area in the MFS group. The first two studies were classified as having a high risk of bias and the last one as an unclear risk by the RoB2 tool.

3.3.4 FOLLOW-UP

We did not find a report of the follow-up time in only one SR²². The unweighted average among all articles was approximately 25 months. Some reviews found a follow-up time of a few hours.

3.3.5 INFECTION

We found an inconsistency between the “results” section and the table in the SR of Avashia et al. 2012¹⁵. The results show that the highest infection rates were for Polyglactin 910/PDS and Silastic Rubber, but the highest index in the table belongs to autologous mandible bone.

Gunarajah and Samman 2013¹⁶ found ten cases of infections in four studies that used porous polyethylene, two cases in studies that used PDS, one case in which titanium mesh was used, one case after using PLLA, P(L/DL)LA 70/30, PLLA/PGA. The SR classified all studies with cases of infection as level 4 of evidence (case series, poor-quality cohort, and case-control studies).

Azzi et al. 2018²⁰ reported only nine cases of infection, all from the same study that evaluated Titanium, Medpor, and Medpor/Titanium. It is unclear which implant(s) were not associated with infection or their proportion.

Oliver et al. 2020²¹ exposed three cases of infection. A retrospective cohort presented two cases after using smooth nylon foil, and a case study described delayed osteomyelitis after using porous polyethylene.

3.3.6 REDUCED OCULAR MOTILITY (ROM) OR ORBITAL SOFT TISSUE ENTRAPMENT (OSTE)

During ROM evaluation, the SR of Avashia et al. 2012¹⁵ described a contradiction between the results section and the table. In the results, we found the highest indices in lyophilized dura and PDS; however, in their table, a demineralized human bone, PDS, and allograft dura are cited as the highest indices.

According to Gunarajah and Samman 2013¹⁶, when evaluating OSTE, five implants remained with OSTE after reconstruction. Some studies, level 4 of evidence, have shown permanence of OSTE after reconstruction with porous polyethylene, polyglactin 910/PDS, allogenic lyophilized dura, and autogenous temporalis fascia, but with resolution > 90% in all. Only one study, level 4, verified the permanence of OSTE in 10 of the initial 60 cases after reconstruction with PDS. We could not find any more information about this paper.

In SR of Dubois et al. 2015¹⁷, when evaluating ROM, two of the four studies classified as RCT, and only one classified as CCT did not find postoperative ROM. An RCT found postoperative ROM when evaluating perforated PDS foil 0.15 mm (n=14) vs. titanium dynamic mesh (n=15), 75% and 88% preoperative and 50% postoperative for both. The other RCT did not provide the data but reported no statistical difference between the nasal septal cartilage vs conchal cartilage groups (n=11 both), pre- or postoperatively.

Singh et al. 2023²⁷ reported ROM in two studies. A retrospective study described the need for revision surgery in two cases that presented mobility restriction and enophthalmos in the manual free-hand-shaped – MFS (n=38). No re-approach was needed in the 3D-printed model-based – 3DP (n=44).

3.3.7 MISCELLANEOUS COMPLICATIONS

Gunarajah and Samman 2013¹⁶ found 18 cases of postoperative orbital dystopia, divided into iliac bone – all from the same study that measured 17 cases pre- and 7-permanence –, porous polyethylene and PDS. The low rate of improvement with PDS is noteworthy (only two of the initial 7). However, it is worth noting that current knowledge contraindicates PDS in the presence of preoperative dystopia. All studies that cited postoperative dystopia were classified as level 4 of evidence.

Wan et al. 2015¹⁸ found two studies that reported postoperative complications in both groups. No significance points to greater involvement in a specific group.

Ramesh et al. 2018¹⁹ very briefly associated two studies, level 2 of evidence, with “inflammation” due to the use of Polydioxanone and Polycaprolactone, but this same citation also referenced study level 4 of evidence, so it was not possible to correlate reliably. They also mentioned 37.5% of delayed inflammation related to PLLA, but we did not find the baseline study of this data. Mucoid cyst formation around Osteomesh was found in a study level of evidence 2, risk of bias 12/16; 13/16.

In Oliver et al. 2020²¹, it was possible to observe four epithelial cysts and four inflammations reported by a case series of four patients, two after hydroxyapatite implantation (after 2.5 and 8 years), one after the use of Titanium* and another after use of Medpor*. The most common complication was graft migration, with 11 cases from the same retrospective cohort that evaluated acrylic or silicone implants. Three cases of graft explantation were reported in two retrospective cohorts, one after the use of PPE + Titanium and the other two after the use of PLLA. Three cases of interest with hematoma were exposed: a retrobulbar hematoma after using u-HA/PLLA, a hematoma after using porous polyethylene, and the last one after using porous polyethylene + Titanium. In none of the cases of hematomas, we found a description in the SR that ruled out possible intervening variables. Except for graft migration (18%), the other complications represent ≤5% of the study sample.

*Note: This statement does not appear explicitly in the text; it is possible to conclude it by connecting the author and title of the paper with another table where the implants used in each study are located.

Bourry et al. 2020²² reported seven patients (0.74%) with “other complications,” which may be hemorrhage, infection, or extrusion of the material. Of these, four were observed after titanium use, two after porous polyethylene, and one after iliac crest bone. After individual analysis, none of the materials presented a higher risk of complications.

Maher et al. 2021²³ cited a list of complications found but did not provide the number of cases or their correlation with the SR groups.

Hartmann et al. 2022²⁴ found complications only in primary studies with patients after tumor resection in their sample. Because they did not meet the eligibility criteria, they were not evaluated.

Singh et al. 2023²⁷ found a study reporting complications, but with patients who did not meet the eligibility criteria of this overview, so they were not evaluated.

4. DISCUSSION

4.1 QUALITY AND BIAS ANALYSIS OF SYSTEMATIC REVIEWS

Due to the heterogeneity of the primary trials, we noticed that some authors of the SRs needed to relativize important factors to gather some information that can be useful to readers. Such relativization impoverishes the methodological quality and, consequently, the reliability of the results.

The results observed in the AMSTAR 2 and ROBIS tools are worrying. Critical question nº7 of AMSTAR 2 received a negative score in all SRs. It questions whether the reviewers provided a list of the papers not included after complete reading and the justifications for doing so. Although unusual, this data is essential for transparency and reduction of selection bias.

During the ROBIS evaluation, we noticed that the major limitations were the inappropriate restriction of eligibility criteria (year of publication and language) and the lack of robustness of its methodology and results to affirm the conclusions presented. With today’s advances in artificial intelligence, we need to rethink the necessity of restricting the language of publication.

4.2 SRS GUIDELINES’ TOOLS

We observed low adherence of SRs authors to use robust tools for assessing methodological quality and analyzing bias in primary studies. We encourage the use of these tools, which, while not ridding us of flaws²⁹, bring greater transparency and less bias.

4.3 OUTCOMES

Figure 4 shows the levels of evidence and strength of recommendation. In general, we identified difficulties with the heterogeneity of data collection and description. Primary studies must provide the raw and measurable data to enable meta-analyses, in addition to statistical significance.

We highlight the insufficient description of complications in general, making it impossible to remove intervening variables.

The studies not mentioned in Figure 4 did not contribute to the discussion with evidence of a systematic review. Due to heterogeneity, most SRs could not synthesize the results and chose to present the raw data of the primary studies. In our interpretation, these results (even if included in the SRs) should be considered as evidence from primary studies, assigning a different weight to the evidence.

4.3.1 ENOPHTHALMOS

We found significant heterogeneity in its description. There seems to be a consensus that enophthalmos is clinical perception or a difference >2 mm between the operated and non-operated orbit ^18,22,25. SRs reported the absence of exophthalmometer data in the primary studies, and we reiterate the importance of such a description.

Because there are many cohorts, we emphasize the relevance of verifying and reporting the existence of “severe” traumas in the sample that may become a confounding factor in the final analysis, given the possibility of lipolysis caused by trauma, which may be independent of reconstruction or implantation.

The SRs of Gunarajah and Samman 2013¹⁶, Dubois et al. 2015¹⁷, Ramesh et al. 2018¹⁹, and Bourry et al. 2020²² converge in concluding the importance of mechanical support in extensive reconstructions.

The results of Murray-Douglass et al. 2022²⁶ point out that 3D-printed implants, contour models, and surgical planning effectively reduce enophthalmos, with no preference for implant biomaterial.

4.3.2 DIPLOPIA

The forms of evaluation/description of diplopia in the studies, when present, are even more heterogeneous. Several studies limit themselves to reporting its existence or not, and few mention in which visual field they find it. We reinforce the suggestion made by Dubois et al. 2015¹⁷ on using Goldman Screens.

We did not find a formal address by SRs about a possible association between the waiting time from trauma to surgery and the presence of postoperative diplopia. We support future authors to verify and report the period between trauma and surgery in cases with diplopia to avoid bias since Volkmann’s contracture can occur regardless of the performance of surgery or the implant used.

Wan et al. 2015¹⁸ suggested that using CAT may provide better results for diplopia. Murray-Douglass et al. 2022²⁶ stated that contour models, moulds, and surgical planning also effectively reduce it.

4.3.3 FRACTURE PATTERNS, SIZE, OR ORBITAL VOLUME

Most studies have cited the importance of defect size, fracture pattern, or combined fractures in the choice of reconstruction method. The description in the primary studies was largely heterogeneous, such as mm, mm², cm, cm², cm³, % of orbital floor, Jaquiéry classification, and mL for volume. This makes it challenging to do a solid correlation. We corroborate the suggestion of Dubois et al. 2015¹⁷: it is necessary to create a three-dimensional model for the classification of defects

A SR by Ramesh et al. 2018¹⁹ concludes, among other topics, that although absorbable implants are suitable for isolated medial wall or floor fractures with intact bony buttresses, PDO, pure PDLLA, and PLLA are unsuitable for orbital reconstructions because they understand that other similar materials present fewer complications. In this SR or the others, we did not find a methodological substrate and quality results supporting these implant contraindications.

In the SRs of Maher et al. 2021²³ and Singh et al. 2023 ²⁷, it was found that manual molding of PSI on 3D printed models helps establish an orbital volume compatible with the non-traumatized orbit

4.3.4 FOLLOW-UP

It is worrisome that some reports involve only a few hours of follow-up. We know that sometimes trauma patients do not return to consultations, but postoperative follow-up for a few months is mandatory for data reliability. In cases where the patient has withdrawn from the treatment before this period, we discourage its maintenance in the sample at the risk of bias^30,31.

4.3.5 INFECTION

None of the implants appeared directly related to higher rates of postoperative infection, and the absence of data made it impossible to provide reliable evidence. The data provided by Avashia et al. 2012¹⁵ claim attention to the 9% of infections found after autologous grafting of the mandible. Although it is insufficient to be considered as evidence, this information should be considered in clinical practice until new evidence is presented.

4.3.6 REDUCED OCULAR MOTILITY (ROM) OR ORBITAL SOFT TISSUE ENTRAPMENT (OSTE)

None of the types of implants appeared to be closely related to this outcome. We did not find a correlation between the waiting interval between the trauma and the surgery and the severity of the trauma. We support future authors to verify and report the period between trauma and surgery in cases with ROM or OSTE to avoid bias since Volkmann’s contracture can occur regardless of the performance of surgery or the implant used.

4.3.7 MISCELLANEOUS COMPLICATIONS

Although several SRs report postoperative complications, few deepen the discussion to reduce the confounding variables.

Even without fulfilling some eligibility criteria, during the screening processes, the SR of  Su et al. 2016 ³² reported a series of epithelial cysts after orbital reconstruction with silicone implants that caught our attention. Although this information does not constitute evidence, it should be considered in clinical practice until new evidence is presented.

5. CONCLUSIONS

Our limitations are available in Table IV, along with their stipulated risks and impacts.

Despite the significant amount of literature on orbital reconstructions, we did not find high-quality evidence. The concern of these results is that some implants have been overvalued or undervalued based on primary studies that have not had methodological validation and definition of statistical weight by the RS community, making these primary studies currently the highest level of evidence available, even if incompatible with this position.

Titanium’s complete biocompatibility defends its safety compared to the other implant materials. Still, the concern regarding the permanence of the implant in the event of a new trauma should be considered, especially in young patients.

For the surgeon, this overview raised a series of important recommendations for the practice (Figure 4). We encourage further randomized controlled trials with a robust research design to increase the quality and reliability of evidence in the field.

REFERENCES

1. Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, et al. Cochrane handbook for systematic reviews of interventions. 2019. https://doi.org/10.1002/9781119536604.
2. Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, Alonso-Coello P, et al. GRADE: An emerging consensus on rating quality of evidence and strength of recommendations. BMJ 2008;336. https://doi.org/10.1136/bmj.39489.470347.ad.
3. Guyatt GH, Oxman AD, Kunz R, Vist GE, Falck-Ytter Y, Schünemann HJ. GRADE: What is “Quality of evidence” and why is it important to clinicians? Chinese J Evidence-Based Med 2009;9. https://doi.org/10.1136/bmj.39490.551019.be.
4. Balshem H, Helfand M, Schünemann HJ, Oxman AD, Kunz R, Brozek J, et al. GRADE guidelines: 3. Rating the quality of evidence. J Clin Epidemiol 2011;64:401–6. https://doi.org/10.1016/j.jclinepi.2010.07.015.
5. The Joanna Briggs Institute. Joanna Briggs Institute Reviewer’s Manual. Joanna Briggs Inst 2014.
6. National Institute for Health and Care Excellence. Developing NICE guidelines: the manual. Process Methods Guid 2014.
7. Fanaroff AC, Califf RM, Lopes RD. High-quality evidence to inform clinical practice. Lancet 2019;394:633–4. https://doi.org/10.1016/S0140-6736(19)31256-5.
8. Masic I, Miokovic M, Muhamedagic B. Evidence Based Medicine – New Approaches and Challenges. Acta Inform Medica 2008;16:219. https://doi.org/10.5455/AIM.2008.16.219-225.
9. Gates M, Gates A, Pieper D, Fernandes RM, Tricco AC, Moher D, et al. Reporting guideline for overviews of reviews of healthcare interventions: Development of the PRIOR statement. BMJ 2022:1–13. https://doi.org/10.1136/bmj-2022-070849.
10. A E, Z F, H H, I I, M K, M O. Mourad Ouzzani, Hossam Hammady, Zbys Fedorowicz, and Ahmed Elmagarmid. Rayyan — a web and mobile app for systematic reviews. Systematic Reviews (2016) 5:210, DOI: 10.1186/s13643-016-0384-4. Abstr 22nd Cochrane Colloq 2016.
11. Shea BJ, Reeves BC, Wells G, Thuku M, Hamel C, Moran J, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ 2017;358:4008. https://doi.org/10.1136/BMJ.J4008.
12. Whiting P, Savović J, Higgins JPT, Caldwell DM, Reeves BC, Shea B, et al. ROBIS: A new tool to assess risk of bias in systematic reviews was developed. J Clin Epidemiol 2016;69:225–34. https://doi.org/10.1016/j.jclinepi.2015.06.005.
13. Moher D, Liberati A, Tetzlaff J, Altman DG, Altman DG, Antes G, et al. Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. PLoS Med 2009;6. https://doi.org/10.1371/journal.pmed.1000097.
14. McGuinness LA, Higgins JPT. Risk-of-bias VISualization (robvis): An R package and Shiny web app for visualizing risk-of-bias assessments. Res Synth Methods 2021;12:55–61. https://doi.org/10.1002/jrsm.1411.
15. Avashia YJ, Sastry A, Fan KL, Mir HS, Thaller SR. Materials used for reconstruction after orbital floor fracture. J Craniofac Surg 2012;23:1991–7. https://doi.org/10.1097/SCS.0b013e31825aada1.
16. Gunarajah DR, Samman N. Biomaterials for repair of orbital floor blowout fractures: A systematic review. J Oral Maxillofac Surg 2013;71:550–70. https://doi.org/10.1016/j.joms.2012.10.029.
17. Dubois L, Steenen SA, Gooris PJJ, Mourits MP, Becking AG. Controversies in orbital reconstruction – I. Defect-driven orbital reconstruction: A systematic review. Int J Oral Maxillofac Surg 2015;44:308–15. https://doi.org/10.1016/j.ijom.2014.12.002.
18. Wan KH, Chong KKL, Young AL. The Role of Computer-Assisted Technology in Post-Traumatic Orbital Reconstruction: A PRISMA-driven Systematic Review. Sci Rep 2015;5:1–8. https://doi.org/10.1038/srep17914.
19. Ramesh S, Hubschman S, Goldberg R. Resorbable Implants for Orbital Fractures: A Systematic Review. Ann Plast Surg 2018;81:372–9. https://doi.org/10.1097/SAP.0000000000001504.
20. Azzi J, Azzi AJ, Cugno S. Resorbable material for pediatric orbital floor reconstruction. J Craniofac Surg 2018;29:1693–6. https://doi.org/10.1097/SCS.0000000000004630.
21. Oliver JD, Saba ES, Gupta N, Hendricks TM, Singh DJ. Alloplastic reconstruction of orbital floor fractures: a systematic review and pooled outcomes analysis. Eur J Plast Surg 2020;43:109–16. https://doi.org/10.1007/s00238-019-01614-x.
22. Bourry M, Hardouin JB, Fauvel F, Corre P, Lebranchu P, Bertin H. Clinical evaluation of the efficacy of materials used for primary reconstruction of orbital floor defects: Meta-analysis. Head Neck 2021;43:679–90. https://doi.org/10.1002/hed.26518.
23. Maher DI, Hall AJ, Gwini SM, Ben Artsi E. Patient-specific Implants for Orbital Fractures: A Systematic Review. Ophthal Plast Reconstr Surg 2022;38:417–24. https://doi.org/10.1097/IOP.0000000000002089.
24. Hartmann A, Kämmerer P, Ortolano LC, Sagheb K, Seiler M. Customised products for orbital wall reconstruction: a systematic review. Br J Oral Maxillofac Surg 2022;60:e702–11. https://doi.org/10.1016/j.bjoms.2021.10.005.
25. Kotecha S, Ferro A, Harrison P, Fan K. Orbital reconstruction: a systematic review and meta-analysis evaluating the role of patient-specific implants. Oral Maxillofac Surg 2023;27:213–26. https://doi.org/10.1007/s10006-022-01074-x.
26. Murray-Douglass A, Snoswell C, Winter C, Harris R. Three-dimensional (3D) printing for post-traumatic orbital reconstruction, a systematic review and meta-analysis. Br J Oral Maxillofac Surg 2022;60:1176–83. https://doi.org/10.1016/j.bjoms.2022.07.001.
27. Singh AK, Khanal N, Chaulagain R, Sharma N, Thieringer FM. Is the Pre-Shaping of an Orbital Implant on a Patient-Specific 3D-Printed Model Advantageous Compared to Conventional Free-Hand Shaping? A Systematic Review and Meta-Analysis. J Clin Med 2023;12. https://doi.org/10.3390/jcm12103426.
28. Abukhder M, Onions E, Flaherty E, Tarassoli S, Hassan MR, Whelan R. A systematic literature review and narrative synthesis on the use of autologous cartilage in the repair of orbital fractures. Ann Med Surg 2024;86:968–74. https://doi.org/10.1097/ms9.0000000000001598.
29. Howick J, Koletsi D, Ioannidis JPA, Madigan C, Pandis N, Loef M, et al. Most healthcare interventions tested in Cochrane Reviews are not effective according to high quality evidence: a systematic review and meta-analysis. J Clin Epidemiol 2022;148:160–9. https://doi.org/10.1016/j.jclinepi.2022.04.017.
30. Von Allmen RS, Weiss S, Tevaearai HT, Kuemmerli C, Tinner C, Carrel TP, et al. Completeness of Follow-Up Determines Validity of Study Findings: Results of a Prospective Repeated Measures Cohort Study. PLoS One 2015;10. https://doi.org/10.1371/JOURNAL.PONE.0140817.
31. Cuzick J. The importance of long-term follow up of participants in clinical trials. Br J Cancer 2022 1283 2022;128:432–8. https://doi.org/10.1038/s41416-022-02038-4.
32. Su Y, Sun J, Fan X. Epithelial cysts associated with alloplastic implants after repair of orbital fractures : a systematic review and four new cases. Br J Oral Maxillofac Surg 2016;54:658–63. https://doi.org/10.1016/j.bjoms.2016.03.028.

Figure 1. Flow diagram, based on PRISMA flowchart, with step-by-step searches and article selection

Figure 2. AMSTAR 2 summary plot [11,14]

Figure 3. ROBIS summary plot [12, 14]

Figure 4. Summary of evidence

Table I. PECOS criteria

Table II. Overall characteristics of the selected reviews

Table III. Analytical characteristics of the selected reviews

Table IV. Limitations

APPENDICES

Appendix 1. PRIOR checklist

Appendix 2. Complete Search Strategy

Appendix 3. Articles excluded after full-text assessment

^[1] Ph.D. candidate in Dental Sciences – Diagnosis and Surgery, at São Paulo State University (Unesp), School of Dentistry, Araraquara, São Paulo. (Pós-graduação Stricto sensu). Oral and Maxillofacial Surgeon from the Federal University of Santa Catarina and Hospital Governador Celso Ramos, Florianópolis, Santa Catarina. (Pós-graduação Lato sensu). ORCID: https://orcid.org/0000-0003-1154-1462. Currículo Lattes: http://lattes.cnpq.br/6476373698108497.

^[2] OMFS (Pós-graduação Lato sensu), PhD (Pós-graduação Stricto sensu) in Diagnosis and Surgery, São Paulo State University (Unesp), School of Dentistry, Araraquara, São Paulo. ORCID: https://orcid.org/0000-0003-3652-3754. Currículo Lattes: http://lattes.cnpq.br/6084018451320572.

^[3] Ph.D. candidate in Dental Sciences – Diagnosis and Surgery, at São Paulo State University (Unesp), School of Dentistry, Araraquara, São Paulo. (Pós-graduação Stricto sensu). Oral and Maxillofacial Surgeon from São Paulo State University (Unesp), School of Dentistry, Araraquara, São Paulo. (Pós-graduação Lato sensu). ORCID: https://orcid.org/0000-0003-4175-6253. Currículo Lattes: http://lattes.cnpq.br/2785523338509611.

^[4] Ph.D. candidate in Dental Sciences – Diagnosis and Surgery, at São Paulo State University (Unesp), School of Dentistry, Araraquara, São Paulo. (Pós-graduação Stricto sensu). Oral and Maxillofacial Surgeon from São Paulo State University (Unesp), School of Dentistry, Araraquara, São Paulo. (Pós-graduação Lato sensu). ORCID: https://orcid.org/0009-0008-0275-6254. Currículo Lattes: http://lattes.cnpq.br/3469266692413850.

^[5] MSc in Oral and Maxillofacial Surgery from São Leopoldo Mandic College, Campinas, São Paulo. (Pós-graduação Stricto sensu). Oral and Maxillofacial Surgeon from São Paulo State University (Unesp), School of Dentistry, Araraquara, São Paulo. (Pós-graduação Lato sensu). ORCID: https://orcid.org/0009-0003-1347-2365. Currículo Lattes: http://lattes.cnpq.br/3241129664353270.

^[6] Associate Professor from São Paulo State University (Unesp), School of Dentistry, Araraquara, São Paulo. PhD in Dental Clinic from the University of Campinas (UNICAMP), Brazil. (Pósgraduação Stricto sensu). MSc in Dental Clinic from the University of Campinas (UNICAMP), Brazil. (Pósgraduação Stricto sensu). ORCID: https://orcid.org/0000-0001-8736-7507. Currículo Lattes: http://lattes.cnpq.br/3933940257808182.

Material received: May 20, 2024.

Peer-approved material: June 17, 2024.

Edited material approved by the authors: July 4, 2024.