Modeling and Mutational Analysis of the Binding Mode for the Multimodal Antidepressant Drug Vortioxetine to the Human 5-HT3A Receptor

5-Hydroxytryptamine₃ (5-HT₃) receptors are ligand-gated ion channels that mediate neurotransmission by serotonin in the central nervous system. Pharmacological inhibition of 5-HT₃ receptor activity has therapeutic potential in several psychiatric diseases, including depression and anxiety. The recently approved multimodal antidepressant vortioxetine has potent inhibitory activity at 5-HT₃ receptors. Vortioxetine has an inhibitory mechanism that differs from classic 5-HT₃ receptor competitive antagonists despite being believed to bind in the same binding site. Specifically, vortioxetine shows partial agonist activity followed by persistent and insurmountable inhibition. We have investigated the binding mode of vortioxetine at the human 5-HT_3A receptor through computational and in vitro experiments to provide insight into the molecular mechanisms behind the unique pharmacological profile of the drug. We find that vortioxetine binds in a manner different from currently known 5-HT_3A orthosteric ligands. Specifically, while the binding pattern of vortioxetine mimics some aspects of both the setron class of competitive antagonists and 5-hydroxytryptamine (5-HT) with regards to interactions with residues of the aromatic box motif in the orthosteric binding site, vortioxetine also forms interactions with residues not previously described to be important for the binding of either setrons or 5-HT such as Val202 on Loop F. Our results expand the framework for understanding how orthosteric ligands drive 5-HT₃ receptor function, which is of importance for the potential future development of novel classes of 5-HT₃ receptor antagonists.