Proposal of pharmacophore model for HIV reverse transcriptase inhibitors: Combined mutational effect analysis,molecular dynamics,molecular docking and pharmacophore modeling study

Abstract

Objectives: Antiretroviral therapy (ART) efficacy is jeopardized by the emergence of drug resistance mutations in HIV, compromising treatment effectiveness. This study aims to propose novel analogs of Effavirenz (EFV) as potential direct inhibitors of HIV reverse transcriptase, employing computer-aided drug design methodologies. Methods: Three key approaches were applied: a mutational profile study, molecular dynamics simulations, and pharmacophore development. The impact of mutations on the stability, flexibility, function, and affinity of target proteins, especially those associated with NRTI, was assessed. Molecular dynamics analysis identified G190E as a mutation significantly altering protein properties, potentially leading to therapeutic failure. Comparative analysis revealed that among six first-line antiretroviral drugs, EFV exhibited notably low affinity with viral reverse transcriptase, further reduced by the G190E mutation. Subsequently, a search for EFV-similar inhibitors yielded 12 promising molecules based on their affinity, forming the basis for generating a pharmacophore model. Results: Mutational analysis pinpointed G190E as a crucial mutation impacting protein properties, potentially undermining therapeutic efficacy. EFV demonstrated diminished affinity with viral reverse transcriptase, exacerbated by the G190E mutation. The search for EFV analogs identified 12 high-affinity molecules, culminating in a pharmacophore model elucidating key structural features crucial for potent inhibition. Conclusion: This study underscores the significance of EFV analogs as potential inhibitors of HIV reverse transcriptase. The findings highlight the impact of mutations on drug efficacy, particularly the detrimental effect of G190E. The generated pharmacophore model serves as a pivotal reference for future drug development efforts targeting HIV, providing essential structural insights for the design of potent inhibitors based on EFV analogs identified in vitro.

Keywords

HIV- pharmacoresistance Effavirenz molecular dynamics computational screening pharmacophore model

Introduction

New Joint United Nations Program on HIV/AIDS (UNAIDS)¹ data on the global human immunodeficiency virus (HIV) response reveals slowing progress in the fight against the HIV pandemic and reduced resources over the past 2 years from COVID-19 and other global crises. This put millions of lives at risk.² This stagnation in progress resulted in about 1.5 million new infections last year; that is more than 1 million above the global targets.³

Currently, there is no permanent cure or vaccine for AIDS.⁴ However, antiretroviral treatment (ARV) remains the only solution to control this deadly epidemic. A remarkable reduction in AIDS-related mortality has been made possible by current combination therapy, called highly active antiretroviral therapy (HAART). Typically, HAART targets multiple steps in the viral replication cycle consisting of a backbone of two nucleoside reverse transcriptase inhibitors (NRTIs) and a third single or boosted drug, which should be an integrase strand transfer inhibitor (INSTI), a nonnucleoside reverse transcriptase inhibitor (NNRTI) or a boosted protease inhibitor (PI).^5–8

The use of combinations of antiretroviral drugs has proven to be remarkably effective in controlling the progression of the disease caused by the HIV and subsequently prolonging survival.⁹ These drugs are considered as an important class in the arsenal of antiretroviral drugs designed to suppress the multiplication of the virus and reduce viral RNA loads in patients’ plasma. Thus, they help to maintain the immune system.^10,11 However, an undesirable consequence of antiretroviral therapy has been the selection and emergence of drug-resistant viral variants. These variants represent a major obstacle to the long-term efficacy of currently available ARVs.¹² This resistance is mainly due to poor adherence to antiretroviral therapy as well as the rapid and high rate of viral replication and the lack of proofreading enzymes to correct errors that occur when the virus converts its RNA to DNA via reverse transcription.¹³ As a result, HIV often makes replication errors. This can lead to mutations that can in turn result in resistance to antiretroviral drugs.¹⁴ Therefore, drug resistance is the main factor behind the failure of current HIV therapies.¹⁵ This concerns not only treated patients, but also people who are naive to ARV and those who are infected by near-resistant strains (primary resistance), which will make this disease more and more difficult.¹⁶

Accordingly, one of the main challenges is to distinguish between mutations that cause antiretroviral resistance from the ones that are neutral, and those that are selectively beneficial to the organisms.¹⁷ This requires an understanding of the effects of missense mutations on the disruption of protein function, which can be induced by modulating their structural properties or by disrupting their interactions with ligands (drugs).¹⁸ The ability to predict drug resistance in HIV reverse transcriptase and protease mutants may be useful in developing more effective and sustainable treatment regimens.¹⁹

The stability and flexibility of these folded viral enzymes (Reverse Transcriptase “RT” and Protease “PROT”) are essential to their biological function. Accordingly, any change in these parameters could affect the efficacy of therapeutic molecules.²⁰ However, mutations can induce radical changes in protein structures.²¹ All of these interrelated changes act synergistically to confer drug resistance while simultaneously maintaining the ability of the virus.²² It follows that a study of the mutational profile to predict the effects of mutations on the protein structures in question is necessary to better understand the mechanisms of antiretroviral resistance.²³ This investigation in turn will lead to the proposal of more efficient molecules. Therefore, many bioinformatics tools have been developed in recent years in silico method to predict the effect of mutations on the stability, flexibility, function and binding energy of protein structures.²⁴ Ranging from empirical sequence-based approaches to rigorous physics-based free energy methods.²⁵

The development of effective HIV reverse transcriptase inhibitors is a critical endeavor in the ongoing battle against HIV/AIDS. In this study, we embark on a computational approach to identify novel analogues of Efavirenz (EFV), a well-established HIV reverse transcriptase inhibitor. This choice is underpinned by a clear rationale and presents several distinct advantages.

The selection of EFV as the starting point for our analogues in this study is grounded in a compelling rationale. EFV is a pivotal component of HAART regimens employed against HIV-1 infection.^26–28 As a NNRTI, it effectively targets the HIV-1 reverse transcriptase enzyme, a key player in the virus’s replication cycle.²⁹ What sets EFV apart is its proven efficacy, particularly in resource-limited settings, where access to treatment options is crucial.^26,27 In comparison to nevirapine (NVP), another NNRTI used in HAART regimens, EFV demonstrates a more robust and durable first-line profile.²⁹ Furthermore, EFV boasts a superior toxicity profile compared to NVP, enhancing patient tolerability.²⁹ The World Health Organization (WHO) has recognized EFV as the preferred first-line NNRTI, underscoring its global importance in HIV treatment.^26,29 Importantly, studies have shown that EFV is less likely to lead to virological failure when compared to NVP, reinforcing its status as a pivotal starting point for the development of improved analogues in our research.^26,29Its robust inhibitory activity and known mechanism of action provide a solid foundation for exploring analogues with enhanced efficacy and reduced side effects By leveraging the existing knowledge base surrounding EFV, we aim to streamline the drug discovery process.

The advantages of this approach are twofold. The selection of EFV as our starting point for analogues is founded upon a robust foundation of attributes. EFV stands as a well-established drug, lauded for its proven track record of efficacy and safety in treating HIV-1 infection.^26,27,30 Distinguished by its non-nucleoside structure, EFV belongs to a distinct drug class from NRTIs and protease inhibitors (PIs) commonly employed in HAART regimens.³¹ Its unique mechanism of action, targeting the HIV-1 reverse transcriptase enzyme, sets it apart.²⁹ Furthermore, EFV’s well-characterized pharmacokinetic profile offers a reference point for designing analogues with improved pharmacokinetic properties.³⁰ Its relatively simple chemical structure provides an ideal canvas for modification, enhancing attributes such as potency, selectivity, and pharmacokinetic properties. EFV’s high binding affinity to the HIV-1 reverse transcriptase enzyme establishes it as an excellent starting point for designing analogues with improved binding capabilities. With its low molecular weight, EFV facilitates the development of analogues with enhanced oral bioavailability and blood-brain barrier penetration.^30,31 Notably, the cost-effective production of EFV underscores its economic viability as a starting point for drug production, cementing its role as a prime candidate for the development of improved HIV treatments.³¹

By predicting drug resistance mutations, researchers can identify the mutations that make certain drugs ineffective and guide the choice of effective therapy, This is particularly important in the case of HIV, where drug resistance has become a severe challenge for treatment of HIV infections.³² Predicting the effects of drug resistance mutations on the structural and functional properties of viral enzymes can also help researchers propose potential inhibitors that can overcome drug resistance.³³

Therefore, the main objective of this study is to predict the effect of mutations conferring resistance to antiretroviral drugs on the structural and functional properties of each of the key viral enzymes of the HIV cycle and then confirm these effects by molecular dynamics simulations. Thus, to propose EFV analogs as potential direct HIV reverse transcriptase inhibitors based on computer-aided drug design through the application of two different approaches including virtual screening and pharmacophore development.

Materials and methods

Nature of the study

This study employs a computational approach utilizing computer-aided drug design methodologies. It encompasses in silico analyses integrating mutational profile studies, molecular dynamics simulations, and pharmacophore development. The investigation primarily focuses on the assessment of HIV reverse transcriptase and associated mutations, aiming to propose novel analogs of EFV as potential direct inhibitors. The computational nature of this research involves predictive modeling and analysis conducted entirely in silico, utilizing computational tools and algorithms to explore molecular interactions and drug-protein dynamics.

First part: Mutational profile study

Mutations collection

Forty seven mutations (SNPs) located on the coding regions of each of the protease and reverse transcriptase genes of Pol³⁴ are described in the literature and proven to be behind drug resistance. They were collected from the HIV French Resistance database which lists all mutations conferring a possible genotypic resistance.³⁵ Then, a screening was performed, according to the calibrated population resistance (CPR) platform,³⁶ to retain the 21 mutations with high-level drug resistance which refers to strains of HIV that are resistant to multiple classes of antiretroviral drugs. Those mutations are distributed according to the three antiretroviral drug classes: two mutations for protease inhibitors (PI), four for nucleotide reverse transcriptase inhibitors (NRTI) and 15 for non-nucleotide reverse transcriptase inhibitors (NNRTI).³⁷

Structures preparation and mutagenesis

The crystallized structures of the two viral proteins of HIV type 1 group M Subtype B were collected in (.pdb) format from the Protein Data Bank (PDB),³⁸ with an ID (1hmv)³⁹ for the viral Reverse Transcriptase (RT) characterized by a resolution of (3.2 Å) and (1Odw)⁴⁰ for the viral protease characterized by a resolution of (2.10 Å).

The 21 mutations obtained after the filtering will be used to generate the mutated models by the Chimera software.⁴¹ These molecular models were prepared with Autodock Tools⁴² (1.5.7) by removing the water molecules and adding the polar hydrogens and Kollman charges and then saved in (.pdbqt) format.

These mutations show drug resistance to the drugs that were collected from PubChem⁴³ format (.SDF) and converted to (.PDB) format with the Open Babel tool.⁴⁴ Using the Vconf tool,⁴⁵ the structures of the drugs were converted from their initial 2D to 3D conformations, then prepared by the Autodock Tool 1.5.7⁴² by adding the Gasteiger loads finally the structures were saved in (.pdbqt) format.

Stability, flexibility, and function predictions

Different methods were used to predict in silico the effect of collected mutations on the stability, flexibility and function of reverse transcriptase and protease structures. For instance, the Site-directed-mutator (SDM) tool⁴⁶ was used to predict the effect of each mutation on the stability of each of the viral proteins already mentioned by specifying the impact that results in stabilizing and destabilizing mutations. DynaMut for the prediction of the impact of mutations on the flexibility of proteins.⁴⁷ Meanwhile, SIFT (Sorting Intolerant From Tolerant) predicts whether an amino acid substitution affects protein function and provides a rapid score-based analysis of protein variants to distinguish deleterious from neutral mutations.⁴⁸

Docking and scoring

To test the impact of these genetic variations on the affinity of the three drug classes with their viral targets, a ligand-protein molecular docking was required. Its main goal is to predict the predominant binding mode(s) of the antiretroviral with the three-dimensional structures of their targets (Reverse transcriptase and viral protease).⁴⁹ Having selected the active site residues of the target protein, the use of Auto Dock tools 1.5.7 allowed the preparation of the grid box. It has a spacing of 1 Å for each macromolecular protein and center coordinates fixed at (X = 38.966, Y = 90.772, Z = 218.363) pour 1hmv et (X = 4.963, Y = 1.179, Z = 18.482) pour 1ODW and a size of (X = 18, Y = 18, Z = 18). The grid settings file was retrieved from the grid menu option. In order to choose ligands with lower affinity scores, Autodock vina⁵⁰ was used to assess the binding affinity and bound conformation of those selected ligands. This tool is designed for protein-ligand docking, using multiple CPUs at a time, making it faster and more accurate and uses the Lamarckian Genetic Algorithm and semi-empirical free energy force field that generates free binding energy of 10 ligand poses after docking. 3D protein-ligand interactions, h-bonds and poses were visualized in PyMol.⁵¹

Structural effect prediction

HOPE⁵² was used to analyze the structural effects of a point mutation (G190E) in a protein sequence (viral reverse transcriptase).

Molecular dynamics simulations

In order to study the impact of the G190E mutation on the structure of the viral reverse transcriptase enzyme, MD simulations were carried out for the wild type and the mutant type for a period of 100 ns using GROMACS (v.2020.4 (53)).⁵³ The protein structure was obtained from the PDB and preprocessed to remove any heteroatoms or water molecules by Pymol. The CHARMM27 force field parameters were assigned to the protein atoms using molecular modeling software in order to define the interactions between the protein and the solvent. A cubic simulation box with an edge length of 1.5 nm was created using the simulation software. The TIP3P water model was used to solvate the protein by adding water molecules to the box. To neutralize the system, an appropriate number of positive and negative ions were randomly placed in the box to reach the desired ion concentration. The system was then subjected to energy minimization using the steepest descent algorithm. This process aimed to relieve any steric clashes or high-energy contacts present in the system, allowing it to settle into a more stable conformation. The equilibration phase consisted of two steps. First, the system was equilibrated at a constant temperature (300 K) using the V-rescale thermostat for a duration of 100 ps. This helped the system reach a consistent temperature and allowed solvent molecules to properly interact with the protein. Next, an equilibration at 1 atm pressure was performed using the Parrinello-Rahman algorithm for another 100 ps. This step ensured that the system achieved a balanced pressure distribution. Following equilibration, production runs were conducted using the NPT ensemble with a time step of 100 ps. These simulations were carried out for an extended period to capture the dynamics and behavior of the protein in the solvent environment. Throughout the simulation, bond constraints were enforced using the LINCS algorithm with a distance cutoff, employing the Verlet method28.

Molecular dynamics analysis

After the production runs, various analysis techniques were applied to extract relevant information from the simulations, The Root-Mean-Square Deviation (RMSD) of atomic coordinates, root mean square fluctuation (RMSF) and radius of gyration (Rg) parameters were calculated using the gmx_rms tool in GROMACS.32.⁵³ These analyses helped to elucidate the behavior and properties of the system under study.

Second part: Proposal of potential inhibitors of Efavirenz

Ligands preparation and filtration

Efavirenz is a medication used to treat the HIV and is a NNRTI.⁵⁴ The drug is available in combination with other antiviral medications, such as emtricitabine and tenofovir disoproxil fumarate, or lamivudine and tenofovir disoproxil fumarate.⁵⁴ It is available as an oral capsule and an oral tablet, with capsules available in 200 mg and 50 mg strengths, and tablets in 600 mg strength.⁵⁴ The manufacturing process for EFV has been detailed in the Active Substance Master File (ASMF) procedure.⁵⁵ Additionally, a Drug Master File (DMF) is available for the EFV active pharmaceutical ingredient (API).⁵⁶ The drug is also available as a USP Reference Standard, which can be used to determine the strength, quality, purity, and identity of the prescribed USP-NF monograph tests and assays.⁵⁷

To identify alternative drug molecules to EFV which is relatively ineffective based on its low binding energy with the viral reverse transcriptase enzyme. Two hundred and sixty two analogues were downloaded from PubChem with a 2D conformation in a format (.sdf) which was converted into 3D and PDB format respectively by the Open Babel and Vconf tools, only 254 molecules are drug-like obeying the rules of Lipinski, Veber and Ghose were screened by the Drulito tool⁵⁸ then an in silico toxicity filtration was performed by the STopTox⁵⁹ resulting in 249 non-toxic molecules,then removal of duplicated structures were performed to keep 200 compounds which were prepared using the Autodock Tool 1.5.7⁶⁰ with the same steps mentioned before.

Docking and scoring

The molecular docking was provided by Autodock Vina.⁶¹ The three-dimensional (3D) visualization of all the interactions has been done by the software: Pymol⁵¹

3D-pharmacophore model building

To search the 3D distances between the common features of the best molecules, a pharmacophoric query method was used; the pharmacophore query was used. It’s based on searching the features like HBD/HBA hydrogen bond donors/acceptors, PI positive ionizable, ARO aromatic ring, Hyd hydrophobic centers in the pharmacophoric map and describing the distance between each point. This part was carried out by using the Molecular Operating Environment (MOE) tool.⁶² Builder option was used to create A MOE database, the energy was optimized and then the alignment was started to generate the pharmacophoric map with the common features.

Results

First part

Study of the mutational profile

Forty seven resistance mutations causing HIV were collected from the HIV French Resistance platform as shown in Figure 1. Then it is filtered by the CPR platform to keep the 21 mutations (Figure 2) compromising a high level of drug resistance to the treatments dispatched according to the three antiretroviral drug classes: two mutations for PI, four for NRTI and 15 for NNRTI.

Figure 1.

Gag Pol mutations distribution. The plot depicts a lollipop graph (1) showing the location of each mutation over the protein domains (Uniprot id P04585).

Figure 2.

Variation of stability and flexibility in mutations of NRTI (a), NNRTI (b) and PI (c) docking results between (Effavirenz-RT) and between (Effavirenz-mutated RT structures).

The study of the mutations’ effect on the physicochemical properties of protein structures (Reverse Transcriptase and Protease) on the three drug classes showed that the four mutations studied altering the NRTI target increases the stability of the protein structure with scores ranging from (.16 to .48). Meanwhile, they decrease the flexibility with scores ranging from (−4.316 to −4.625) as is shown in Figure 2(a).

Mutations that affect the target of NNRTI are represented in Figure 2(b). The 15 mutations decrease the flexibility with values varying between (−4.032 and −4.987) while 12 of them decrease the stability against three that make the protein structure more stable with a score varying between (−2.51 and .320).

While the two mutations affecting the PI target shown in Figure 2(c), they decrease the stability of the protein structure in question with values between −1.42 and −1.49 and increase its flexibility with scores of .060 and .979 successively for I47 A and L76 V.

The study of the mutations’ impact on function showed that 67% of them are deleterious to protein function, whereas 33% are tolerable (Figure 3).

Figure 3.

Distribution of mutations deleterious (pink) and neutral (blue).

Docking

After the generation of the 21 mutated models, molecular docking was performed between the reference protein structures and the drugs where they show drug resistance, and then between the generated mutated structures and the drugs.

The docking with NRTI showed that the four mutations studied (Q151M-Y115F-M184V and M184I) decrease the binding energy with the targeted drug be it Zidovudine (ZDV) Abacavir (ABC), Lamivudine (3TC) or Tenofovir (FTC) (Figure 4(a)) and (Supplemental Table S1).

Figure 4.

Binding energy of the three drug classes; NRTI (a), NNRTI (b) and PI (c).

Figure 4(b) and (Supplemental Table S2). shows that all the mutated models of the reverse transcriptase have a lower binding affinity than the wild type with the NNRTIs: EFV and NVP. The graph shows that G190E is the mutation that decreases most this affinity with EFV, which was a value of −4 and became −1.2.

Regarding the molecular docking performed for the two mutations that alter the IP target (I47A and L76V) shown in Figure 4(c) and (Supplemental Table S3), their affinity energies with the drug Lopinavir/Ritonavir (LVP/r) which is (−6) decrease compared to the wild type which was (−5.5).

Figure 5 below provides a visualization of the schematic structures generated by the Hope tool, depicting the original (G) amino acid (left) and its mutant counterpart (E) (right). In each representation, the backbone, uniformly shared by both amino acids, is highlighted in red, while the distinctive side chains, unique to each amino acid, are displayed in black. Additionally, Figure 6 showcases the post-docking visualization, presenting the overlaid 3D structures of the wild-type and G190E-mutated RTase. This comparative view offers insights into the spatial alterations resulting from the mutation within the RTase.

Figure 5.

The schematic structures of the original (left) and the mutant (right) amino acid.

Figure 6.

Comparative 3D Structures of Wild-Type (a) and G190E-Mutated RTase (b): Post-Docking visualization.

Molecular dynamic analysis

The results of molecular dynamics simulations reveal significant differences between the wild type and the mutated type in terms of several evaluated parameters, including RMSD, RMSF, and GR.

Regarding RMSD, which measures the average deviation of atomic positions from the reference structure, it is significantly higher in the G190E mutated type (.69 ± .16) compared to the wild type (.37 ± .05), indicating greater flexibility or structural deviation (Figure 7(a)).

Figure 7.

Analysis of molecular dynamics simulations done in 100 ns for wild-type and mutated reverse transcriptase protein; (a) The RMSD analysis of wt-RT (black) and mt-RT (red). (b) The RMSF of each wt-RT and mt-RT according to residue numbers. (c) The radius of gyration of the wild and mutated protein calculated at the two simulations.

Furthermore, RMSF, which assesses local fluctuations of atoms in a protein, shows higher values in the mutated type (.34 ± .13) compared to the wild type (.21 ± .07) for several specific residues. For instance, residues (53, 250, 450, 471, and 539) in the mutated type exhibit higher RMSF values than their counterparts in the wild type, suggesting increased mobility or flexibility at these residues (Figure 7(b)).

Regarding Rg, which measures the overall compactness of a protein, a significant increase is observed in the mutated type (3.55 ± .12) compared to the wild type (3.24 ± .05), indicating greater expansion or size in the conformation of the mutated type (Figure 7(c)).

These results clearly indicate that the mutated type exhibits significant structural variations compared to the wild type.

Second part

The results of the first part served as a matrix on which this second part is based. It should be known that the G190E mutation is deleterious for the viral enzyme function, presents a high level of drug resistance, destabilizing and decreasing the flexibility of the protein structure, negatively affects the binding energy with the therapeutic target and decreases the number of bonds with the drug which is EFV. To confirm the results obtained previously, another molecular docking was performed between the six first-line antiretroviral drugs and the reverse transcriptase showed that EFV is the drug which presents a low docking score (−3.9) and therefore a low binding energy with the viral RT target (Figure 8).

Figure 8.

Affinity energy between first-line antiretroviral drugs and reverse transcriptase.

Since EFV has the lowest affinity energy compared to other antiretrovirals, we opted to look for one or more analogues that will be better in terms of binding energy even in the presence of mutations.

Virtual screening

For this fact, 262 similar structures of the EFV according to the Tanimoto score were collected and filtered by Drulito in order to keep 254 drug substances respecting both the rules of Veber, Lipinski and Ghose, then their toxicity was evaluated by STopTox in order to keep 249 non-toxic molecules, then 200 molecules after the elimination of the duplicated structures as is shown in Figure 9.

Figure 9.

Virtual screening and filtration of EFV analogues.

Docking and scoring

After all the filtrations, 200 structures were obtained and subjected to docking. The results, as illustrated in Supplemental Table S1, reveal that 61 structures exhibit affinities ranging from (−5.5 to −4.1), surpassing the affinity of EFV, which was recorded at (−4). Subsequently, these structures underwent a second round of docking with the mutated structure induced by the G190E mutation. Supplemental Table S4 represents the structures with higher binding affinity than EFV and highlights the differences in binding energy between their wild-type and mutant forms. Interestingly, among the 61 structures, 11 retained the same affinity as the wild-type structure, while 12 exhibited an enhanced affinity. In contrast, 48 structures experienced a reduction in binding energy.

To gain further insights into the characteristics of the 12 candidate molecules, details such as names, formulas, molecular weights, and 3D structures are provided in Supplemental Table S5. Additionally, the results of the ADME studies for these 12 structures are presented in Supplemental Table S6 in the supplemental material.

3D- pharmacophore modelling

The pharmacophoric map of the best 12 molecules was generated. Using MOE software, the common features of these compounds are shown in Figure 9. The map shows a simplified view with the aligned 12 ligands (Figure 10(a)) and without ligands (Figure 10(b)). The 3D-Pharmacophoric model consists of a three common features; The aromatic ring may be a player key in the binding to the target protein. The three features (F1-F2-F3) connected with a distance of 3.16 Å, 3.46 Å and 4.34 Å respectively (Figure 10(c)).

Figure 10.

3D Pharmacophoric map of the best ligands. (a), a simplified view of the model with aligned ligands. (b), a simplified view without aligned ligands and (c), the distance between the common features found.

Discussion

In this study, all mutations responsible for possible therapeutic failure were collected and filtered. This was done in accordance with the level of drug resistance against the antiretroviral drugs. For this reason, several platforms were used to select only those with a high level of drug resistance and categorize them according to three drug classes (four for NRTI- 15 for NNRTI and two for PI).

Following this, a global analysis was performed to study the effect of mutations on the structural and functional properties of the viral enzymes (reverse transcriptase and protease). These properties include stability, flexibility, as well as function and affinity energy of the two viral proteins.

The four mutations that present a high level of resistance to nucleotide inhibitors of reverse transcriptase are all deleterious of the proteins’ function. They increase stability and decrease flexibility. Therefore, there will induce an increase of rigidity. As a result, this may decrease the conformations that the protein can take and then will be unable to interact with the ligand because of its probable loss of the proper conformation for this binding and consequently a loss of function.^63,64 Accordingly, this may explains the severity of these mutations in patients and their direct impact on the protein.

Concerning the 15 mutations altering the target of the NNRTI drug class, they are dispatched between nine deleterious mutations and six neutral mutations. Then into 13 mutations, which decreases stability, and 2 that increase it. While, the totality of the mutations decrease flexibility. Thus, the majority of these mutations decrease the two parameters of stability and flexibility, which may explain the fact that these mutations act in an indirect way on the protein.⁶⁵

As for the two mutations that alter the viral protease, they are deleterious since they decrease the stability and increase the flexibility. A thing that favors an increase in the conformations that the protein can take. In that, it can decrease the chances that the protein interacts with its target.⁶⁶ Therefore, a probable loss of function, followed by a direct impact of these mutations on the protein.⁶⁴

Consequently, these effects may explain the importance of each drug class and the severity of the syndrome likely after alteration of their targets by mutations. It also entitles us to suggest that the mutations that confer drug resistance to NNRTIs are less severe in comparison with those involving NRTIs and PIs.

It follows that the molecular docking between seven drugs and the generated mutated models was performed on three categories: NNRTI-NRTI-PI. The results showed that the totality of the mutations decreased the binding energy and the number of interactions between ligands and receptors. The majority revealed a significant decrease in terms of docking score provided to maintain a better binding with the target. Thus, from this we can deduce that these mutations act negatively on the physicochemical as well as on functional properties of the two viral enzymes in question.

Each amino acid has its own specific size, charge, and hydrophobicity-value. The original wild-type residue and newly introduced mutant residue often differ in these properties: in our mutation G190E, the mutant residue is bigger than the wild-type residue and had a NEUTRAL charge, the mutant residue charge is NEGATIVE. As well as the wild-type residue is more hydrophobic than the mutant residue. The mutant residue introduces a charge in a buried residue which can lead to protein folding problems.

In terms of structure, the wild-type residue is a glycine, the most flexible of all residues. This flexibility might be necessary for the protein’s function. Mutation of this glycine can abolish this function. The torsion angles for this residue are unusual. only glycine is flexible enough to make these torsion angles, mutation into another residue will force the local backbone into an incorrect conformation and will disturb the local structure. The wild-type residue was buried in the core of the protein. The mutant residue is bigger and probably will not fit.

In molecular dynamics, RMSD stands for ‘Root-Mean-Square Deviation’ and is used to measure the similarity or difference between two molecular structures. Our simulation revealed a high RMSD of the mutated type (.69 ± .16) compared to the wild-type (.37 ± .05), indicating significant differences in the molecular structures of the two types. RMSD is calculated by measuring the distance between each atom in the two structures and calculating the average of the squares of these distances.⁶⁷ A high RMSD suggests that the atoms in the two structures are significantly displaced from each other. In the context of our mutation, this may indicate that the mutation introduces important conformational or structural changes in the reverse transcriptase. Mutations can lead to modifications in the spatial conformation of atoms, potentially affecting molecular interactions, structural stability, or functional properties of the molecule.⁶⁸

As for RMSF, it stands for ‘Root-Mean-Square Fluctuation’ and is used to evaluate local fluctuations of atoms in a molecular structure. The RMSF’s value of the G190E mutated type (.34 ± .13) is higher compared to the wild-type ss (.21 ± .07), indicating that the atoms in the mutated type tend to fluctuate more than in the wild-type.⁶⁹ RMSF is calculated by measuring the deviations (fluctuations) of average atomic positions from their mean position during a molecular dynamics simulation.⁷⁰ The elevated RMSF value may suggest greater flexibility or local instability in the structure of the mutated protein or molecule compared to the wild-type. Mutations can disrupt atom interactions, resulting in increased mobility in specific regions of the protein.

The last evaluated parameter is Rg, which stands for ‘Radius of Gyration’ and is a measure of the overall compactness of a molecular structure.⁷¹ The Rg of the mutated type obtained from our results (3.55 ± .12) is higher compared to the wild-type (3.24 ± .05), indicating that the molecular structure of the mutated type is more expanded or looser than that of the wild-type.Rg is calculated by determining the average distance between each atom and the center of mass of the molecule, providing an estimation of the size and compactness of the structure.⁷² The elevated Rg values obtained from the simulations allow us to suggest that the mutated type exhibits greater flexibility or conformational expansion compared to the wild-type. The G190E mutation may disrupt atomic interactions and lead to conformational changes that broaden or extend the overall structure of the molecule.”

The results of the mutational effects prediction, the molecular docking and the molecular dynamics simulations showed that the deleterious mutation G190E negatively alters all the parameters tested in this study. Thus, it presents a high level of resistance to EFV, which can help us to search for EFV analogues, which may be subject to drug repositioning as a solution to overcome the problem of antiretroviral resistance and so to combat treatment failure.

Docking between the six first-line ARVs (ZDV, FTC, 3TC, ABC, NVP and EFV) and their target, the viral reverse transcriptase enzyme, was performed and resulted in a binding energy varying between −5.1 and −3.9, whose EFV presented the low score. Based on what has been reported and on several articles in the literature^7,73–75 that confirmed the resistance to EFV and subsequently its ineffectiveness, through our study, we tried to propose in silico EFV analogues that can substitute EFV later.

A virtual screening analysis was conducted to collect the similar structures of non-toxic EFV, which respect Lipinski, Veber and Ghose rules and can be proposed as analogues that will substitute EFV, 61 molecules resulted in these criteria of which the best was 4-(2-cyclopropylethynyl)-4-(trifluoromethyl)-1H-3,1-benzoxazin-2-one with a higher affinity than EFV.

The second docking was performed between the 61 structures predicted as EFV analogues and the G190E-mutated protein structure, the results are divided between structures altered and others not altered by the mutation in question, resulting in 23 structures dispatched between 11 that do not affect the binding energy with the mutated protein structure and 12 that improve it. Collectively, the best structure that significantly increases the binding score with the mutated structure from (−4.2) to (−5.5) bearing the scientific nomenclature 4H-3,1-Benzoxazin-2-one, 1,2-dihydro-6-chloro-4,4-dimethyl-.

Based on the structure of these 12 compounds, the pharmacophoric map showed that the hydrophobic Hyd groups and the aromatic Aro regions play a vital role in their binding to the protein. Aro groups enhances the interaction between ligands and the active site of the target and should be considered as crucial structural features. We hope that our contribution can offer a suitable pharmacophore for further studies.

Absolutely, recognizing and discussing the limitations of a study is essential to providing a comprehensive understanding of its scope and potential constraints. for this we will highlight the following limitations:

1. Simplification in Computational Modeling: Computational models simplify complex biological systems, providing valuable insights but not fully replicating real biological intricacies. Discrepancies between computational predictions and actual experimental outcomes need addressing.

2. Dependence on Available Data and Assumptions: The study relies on existing data, databases, and assumptions, potentially impacting prediction accuracy due to limitations or biases in structural and mutational data.

3. Lack of Experimental Validation: The proposed EFV analogs and pharmacophore model lack experimental validation. In vitro or in vivo experiments are essential to verify the efficacy and safety of these analogs as HIV inhibitors.

4. Scope and Generalization: Limitations exist in generalizing findings, especially when focusing on specific HIV mutations or strains, potentially restricting applicability to broader contexts.

Conclusion and perspective

In pursuit of identifying potential direct inhibitors of HIV reverse transcriptase akin to EFV analogues, this study employed in silico approaches. Through meticulous evaluation, we successfully assessed the impact of mutations within the NRTI, NNRTI, and PI drug classes on the stability, flexibility, function, and binding energies of both protease and reverse transcriptase proteins. Subsequently, a pharmacophore model was generated to discern shared characteristics among the most promising ligands. Notably, our findings revealed ligands with lower affinities than the reference ligand EFV in both wild-type and mutant versions.

This pharmacophore model represents a pivotal milestone, serving as a foundational reference for future investigations. It elucidates common structural features essential for effective binding, paving the way for the development of potent drugs targeting HIV. By identifying key characteristics shared among these ligands, this model provides invaluable insights into the design of novel compounds with enhanced efficacy against diverse HIV strains. Moreover, it underscores the potential for further in vitro studies to validate and refine these ligands, offering a promising trajectory for the development of next-generation HIV therapeutics.

In summary, our study not only contributes to the understanding of the impact of mutations on drug-protein interactions but also lays the groundwork for the rational design of efficacious anti-HIV compounds, thereby advancing the pursuit of combating drug resistance and enhancing treatment strategies for HIV infection.

Supplemental Material

Supplemental Material - Proposal of pharmacophore model for HIV reverse transcriptase inhibitors: Combined mutational effect analysis, molecular dynamics, molecular docking and pharmacophore modeling study

Supplemental Material for Proposal of pharmacophore model for HIV reverse transcriptase inhibitors: Combined mutational effect analysis, molecular dynamics, molecular docking and pharmacophore modeling study by Azzeddine Annan, Noureddine Raiss, Sanae Lemrabet, Nezha Elomari, El Harti Elmir, Abdelkarim Filali-Maltouf, Leila Medraoui, and Hicham Oumzil in International Journal of Immunopathology and Pharmacology

Abbreviations

ABC

Abacavir

ART

antiretroviral therapy

ARV

antiretroviral

EFV

Efavirenz

FTC

Emtricitabine

HAART

highly active antiretroviral therapy

INSTI

integrase inhibitor

NNRTI

non-nucleoside reverse transcriptase inhibitor

NRTI

nucleoside reverse transcriptase inhibitors

NVP

Nevirapine

Reverse Transcriptase

3 TC

lamivudine

ZDV

zidovudine

Footnotes

Acknowledgements

The authors would like to thank the virology department personnel especially the HIV diagnosis labs.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iDs

Azzeddine Annan

Noureddine Raiss

Supplemental Material

Supplemental material for this article is available online.

References

UNAIDS . 2023 World AIDS Day — Let Communities Lead. Available at: https://www.unaids.org/en/2023-world-aids-day

UNAIDS . Full report — In Danger: UNAIDS Global AIDS Update 2022. Available at: https://www.unaids.org/en/resources/documents/2022/in-danger-global-aids-update

UNAIDS . Global HIV and AIDS Statistics — Fact sheet. Available at: https://www.unaids.org/en/resources/fact-sheet

Menéndez-Arias

(2013) Molecular basis of human immunodeficiency virus type 1 drug resistance: Overview and recent developments. Antivir Res 98(1): 93–120.

Günthard

Aberg

Eron

International Antiviral Society-USA Panel

et al. (2014) Antiretroviral treatment of adult HIV infection: 2014 recommendations of the international antiviral society–USA panel. JAMA 312(4): 410–425.

Brechtl

Breitbart

Galietta

Krivo

Rosenfeld

(2001) The use of highly active antiretroviral therapy (HAART) in patients with advanced HIV infection: impact on medical, palliative care, and quality of life outcomes. J Pain Symptom Manag 21(1): 41–51.

Bacheler

Jeffrey

Hanna

et al. (2001) Genotypic correlates of phenotypic resistance to efavirenz in virus isolates from patients failing nonnucleoside reverse transcriptase inhibitor therapy. J Virol 75(11): 4999–5008.

Shafer

Vuitton

(1999) Highly active antiretroviral therapy (Haart) for the treatment of infection with human immunodeficiency virus type 1. Biomed Pharmacother. 1 mars 53(2): 73–86.

Palella

Delaney

Moorman

et al. (1998) Declining morbidity and mortality among patients with advanced human immunodeficiency virus infection. HIV Outpatient Study Investigators. N Engl J Med 338(13): 853–860.

10.

Arts

Hazuda

(2012) HIV-1 antiretroviral drug therapy. Cold Spring Harbor perspectives in medicine 2(4): a007161.

11.

WHO . Principaux repères sur le VIH/sida. Available at: https://www.who.int/fr/news-room/fact-sheets/detail/hiv-aids

12.

Desai

Iyer

Dikshit

(2012) Antiretroviral drugs: critical issues and recent advances. Indian J Pharmacol 44(3): 288–298.

13.

Choi

(2012) Viral polymerases. Advances in experimental medicine and biology 726: 267–304.

14.

Tang

Shafer

(2012) HIV-1 antiretroviral resistance. Drugs 72(9): e1–e25.

15.

Zdanowicz

(2006) The pharmacology of HIV drug resistance. Am J Pharmaceut Educ 70(5): 100.

16.

Qian

Yuan

et al. (2017) Drug resistance mutation profiles of the drug-naïve and first-line regimen-treated HIV-1-infected population of Suzhou, China. Virol Sin 32(4): 271–279.

17.

Irwin

Renzette

Kowalik

Jensen

(2016) Antiviral drug resistance as an adaptive process. Virus evolution 2(1): vew014.

18.

Malhotra

Alsulami

Heiyun

et al. (2019) Understanding the impacts of missense mutations on structures and functions of human cancer-related genes: a preliminary computational analysis of the COSMIC Cancer Gene Census. PLoS One 14(7): e0219935.

19.

Bokharaei-Salim

Esghaei

Khanaliha

et al. (2020) HIV-1 reverse transcriptase and protease mutations for drug-resistance detection among treatment-experienced and naïve HIV-infected individuals. PLoS One 15(3): e0229275.

20.

Ali

Bandaranayake

Cai

et al. (2010) Molecular basis for drug resistance in HIV-1 protease. Viruses 2(11): 2509–2535.

21.

Frontiers . Influence of Disease-Causing Mutations on Protein Structural Networks. Available at: https://www.frontiersin.org/articles/10.3389/fmolb.2020.620554/full

22.

Vere Hodge

Field

(2011) General mechanisms of antiviral resistance. Genet Evol Infect Dis 339–362.

23.

Mata-Munguía

Escoto-Delgadillo

Torres-Mendoza

et al. (2014) Natural polymorphisms and unusual mutations in HIV-1 protease with potential antiretroviral resistance: a bioinformatic analysis. BMC Bioinf 15: 72.

24.

Pandurangan

Blundell

(2020) Prediction of impacts of mutations on protein structure and interactions: SDM, a statistical approach, and mCSM, using machine learning. Protein Sci : A Publication of the Protein Society 29(1): 247–257.

25.

Steinbrecher

Zhu

Wang

, et al. (2017) Predicting the effect of amino acid single-point mutations on protein stability-large-scale validation of MD-based relative free energy calculations. J Mol Biol 429(7): 948–963.

26.

Hill

Pozniak

(2016) How can we achieve universal access to low-cost treatment for HIV? Journal of virus eradication 2(4): 193–197.

27.

Castillo-Mancilla

Campbell

(2012) Comparative effectiveness of efavirenz-based antiretroviral regimens in resource-limited settings. Journal of Comparative Effectiveness Research 1(2): 157–170.

28.

McDonagh

Lau

Alvarellos

, et al. (2015) PharmGKB summary: efavirenz pathway, pharmacokinetics. Pharmacogenetics Genom 25(7): 363–376.

29.

Pillay

Ford

Shubber

, et al. (2013) Outcomes for efavirenz versus nevirapine-containing regimens for treatment of hiv-1 infection: a systematic review and meta-analysis. PLoS One 8(7): e68995. Available at: https://journals.plos.org/plosone/article?id=10.1371%2Fjournal.pone.0068995

30.

PMC . PharmGKB Summary: Efavirenz Pathway, Pharmacokinetics (PK). Available at: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4461466/

31.

Bautista

(2023) Various synthetic pathways towards efavirenz and its analogs; the replacement of the side chain. Selected Honors Theses 176.

32.

Weber

Harrison

(2014) Prediction of HIV drug resistance from genotype with encoded three-dimensional protein structure. BMC Genomics 15: S1. Available at: https://bmcgenomics.biomedcentral.com/articles/10.1186/1471-2164-15-S5-S1

33.

Wiley . Predicting Drug-Resistant Mutations of HIV Protease - Ishikita - 2008 - Angewandte Chemie International Edition. Available at: https://onlinelibrary.wiley.com/doi/10.1002/anie.200704178

34.

Yahi

Tamalet

Tourrès

et al. (1999) Mutation patterns of the reverse transcriptase and protease genes in human immunodeficiency virus type 1-infected patients undergoing combination therapy: survey of 787 sequences. J Clin Microbiol 37(12): 4099–4106.

35.

Tables of Rules. Available at: https://hivfrenchresistance.org/hiv-french-resitance-tables-of-rules/

36.

Gifford

Liu

Rhee

, et al. (2009) The calibrated population resistance tool: standardized genotypic estimation of transmitted HIV-1 drug resistance. Bioinformatics 25(9): 1197–1208.

37.

Stanford University . HIVdb Program: Mutations Analysis - Stanford Coronavirus Antiviral and Resistance Database. Available at: https://hivdb.stanford.edu/hivdb/by-patterns/

38.

Burley

Berman

Kleywegt

Markley

Nakamura

Velankar

(2017) Protein Data Bank (PDB): the single global macromolecular structure archive. In: Wlodawer

Dauter

Jaskolski

, (eds) Protein Crystallography: Methods and Protocols. New York, NY: Springer, 627–641.

39.

Rodgers

Gamblin

Harris

, et al. (1995) The structure of unliganded reverse transcriptase from the human immunodeficiency virus type 1. PNAS 92(4): 1222–1226. Available at: https://www.pnas.org/doi/10.1073/pnas.92.4.1222

40.

Protein and Peptide Letters. Available at: https://books.google.com/books/about/Protein_Peptide_Letters.html?hl=fr&id=DxAkDyXefJ4C

41.

Chen

Huang

Ferrin

(2015) RRDistMaps: a UCSF Chimera tool for viewing and comparing protein distance maps. Bioinformatics 31(9): 1484–1486.

42.

Morris

Huey

Lindstrom

et al. (2009) AutoDock4 and AutoDockTools4: automated docking with selective receptor flexibility. J Comput Chem 30(16): 2785–2791.

43.

Kim

Thiessen

Bolton

, et al. (2016) PubChem substance and compound databases. Nucleic Acids Res 44(D1): D1202–D1213.

44.

O’Boyle

Banck

James

Morley

Vandermeersch

Hutchison

(2011) Open Babel: an open chemical toolbox. J Cheminf 3(1): 33.

45.

VeraChem LLC . VConf. Available at: https://www.verachem.com/products/vconf/

46.

Worth

Preissner

Blundell

(2011) SDM—a server for predicting effects of mutations on protein stability and malfunction. Nucleic Acids Res 39: W215–W222.

47.

Rodrigues

Pires

Ascher

(2018) DynaMut: predicting the impact of mutations on protein conformation, flexibility and stability. Nucleic Acids Res 46(W1): W350–W355.

48.

SIFT . Predict effects of nonsynonmous/missense variants. Available at: https://sift.bii.a-star.edu.sg/

49.

Tarasova

Poroikov

Veselovsky

(2018) Molecular docking studies of HIV-1 resistance to reverse transcriptase inhibitors: mini-review. Molecules 23(5): 1233.

50.

Trott

Olson

(2010) AutoDock Vina: improving the speed and accuracy of docking with a new scoring function, efficient optimization, and multithreading. J Comput Chem 31: 455–461.

51.

PyMOL

pymol.org. Available at: https://pymol.org/2/

52.

Venselaar

te Beek

Kuipers

Hekkelman

Vriend

(2010) Protein structure analysis of mutations causing inheritable diseases. An e-Science approach with life scientist friendly interfaces. BMC Bioinf 11: 548.

53.

Bauer

Hess

Lindahl

. GROMACS 2022.4 Source code. Available at: https://zenodo.org/record/7323393

54.

Yee

Preuss

(2023) Efavirenz. Treasure Island, FL: StatPearls Publishing.

55.

European Medicines Agency . efavirenz-teva-epar-public-assessment-report_en.pdf. Available at: https://www.ema.europa.eu/en/documents/assessment-report/efavirenz-teva-epar-public-assessment-report_en.pdf

56.

DMF . PharmaCompass.com. Available at: https://www.pharmacompass.com/us-drug-master-files-dmfs/efavirenz

57.

Efavirenz United States Pharmacopeia (USP) . Reference Standard | 154598-52-4. Available at: http://www.sigmaaldrich.com/

58.

Mignani

Rodrigues

Tomas

, et al. (2018) Present drug-likeness filters in medicinal chemistry during the hit and lead optimization process: how far can they be simplified? Drug Discovery Today 23(3): 605–615. Available at: https://www.sciencedirect.com/science/article/abs/pii/S1359644617304671

59.

Borba

JVB

Alves

Braga

et al. (2022) STopTox: an in silico alternative to animal testing for acute systemic and topical toxicity. Environ Health Perspect 130(2): 27012.

60.

Center for Computational Structural Biology . mgl-admin. Homepage. Available at: https://ccsb.scripps.edu/mgltools/

61.

Center for Computational Structural Biology . mgl-admin. Homepage. Available at: https://vina.scripps.edu/

62.

Chemical Computing Group (CCG) . Research. Available at: https://www.chemcomp.com/Research-Citing_MOE.htm

63.

Mamonova

Glyakina

Galzitskaya

Kurnikova

(2013) Stability and rigidity/flexibility—two sides of the same coin? Biochim biophys acta BBA - proteins proteomics. Biochim Biophys Acta 1834(5): 854–866.

64.

Karshikoff

Jelesarov

(2008) Salt bridges and conformational flexibility: effect on protein stability. Biotechnol Biotechnol Equip 22(1): 606–611.

65.

Smith

Sternberg

MJE

Bates

(2005) The relationship between the flexibility of proteins and their conformational states on forming protein–protein complexes with an application to protein–protein docking. J Mol Biol 347(5): 1077–1101.

66.

Mansouri

El Haddoumi

Bendani

, et al. (2023) In silico analyses of all STAT3 missense variants leading to explore divergent AD-HIES clinical phenotypes. Evolutionary Bioinformatics 19. Available at: https://journals.sagepub.com/doi/epub/10.1177/11769343231169374

67.

Gogoi

Chowdhury

Goswami

et al. (2021) Identification of potential plant-based inhibitor against viral proteases of SARS-CoV-2 through molecular docking, MM-PBSA binding energy calculations and molecular dynamics simulation. Mol Divers 25(3): 1963–1977.

68.

Wang

Alekseenko

Kozakov

Miao

(2019) Improved modeling of peptide-protein binding through global docking and accelerated molecular dynamics simulations. Front Mol Biosci 6: 112. Available at: https://www.frontiersin.org/articles/10.3389/fmolb.2019.00112

69.

Zhang

Sun

Dong

Bao

Lin

(2022) Targeted regulation of pulsed electric field (PEF) treatment on responsive amino acids based on the molecular dynamic simulation. Innovat Food Sci Emerg Technol 82: 103197.

70.

Elham

Mohammad

Fahimeh

(2021) Comparative investigation of R213G mutation in DNA-binding domain of P53 protein via molecular dynamics simulation. Journal of Health and Biomedical Informatics 8(100325): 105–116.

71.

Yamamoto

Akimoto

Mitsutake

Metzler

(2021) Universal relation between instantaneous diffusivity and radius of gyration of proteins in aqueous solution. Phys Rev Lett 126(12): 128101.

72.

Pirhadi

Amani

(2020) Molecular dynamics simulation of self-assembly in a nanoemulsion system. Chem Pap 74(8): 2443–2448.

73.

Marzolini

Telenti

Decosterd

Greub

Biollaz

Buclin

(2001) Efavirenz plasma levels can predict treatment failure and central nervous system side effects in HIV-1-infected patients. AIDS 15(1): 71–75.

74.

Jennings

Kellermann

Spinelli

et al. (2022) Drug resistance, rather than low tenofovir levels in blood or urine, is associated with tenofovir, emtricitabine, and efavirenz failure in resource-limited settings. AIDS Res Hum Retrovir 38(6): 455–462.

75.

Costa

Vale

(2022) Efavirenz: history, development and future. Biomolecules 13(1): 88.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.73 MB