Antibody–drug conjugates (ADC) have emerged as one of the pillars of clinical disease management in oncology. The biggest hurdle to widespread development and application of ADCs has been a narrow therapeutic index. Advances in antibody technologies and formats as well as novel linker and payload chemistries have begun to facilitate structural improvements to ADCs. However, the interplay of structural characteristics with physiologic and pharmacologic factors determining therapeutic success has garnered less attention. This review elaborates on the pharmacology of ADCs, the pathophysiology of cancerous tissues, and the reciprocal consequences on ADC properties and functions. While most currently approved ADCs utilize either microtubule inhibition or DNA damage as primary mechanisms of action, we present arguments to expand this repertoire and highlight the need for payload mechanisms that exploit disease-specific vulnerabilities. We promote the idea that the choice of antibody format, targeting antigen, linker properties, and payload of an ADC should be deliberately fit for purpose by taking the pathophysiology of disease and the specific pharmacology of the drug entity into account, thus allowing a higher probability of clinical success.