In recent years, there has been a growing recognition that P values, albeit useful in reporting data analysis results, have often been misused or misinterpreted in biomedical research. The emergence of big health data such as genomics data and electronic health records, sometimes combined with inadequate experimental design, has exacerbated this problem, which has become a major cause of the ongoing crisis in reproducibility in biomedical research. We aim to shed light and raise awareness of common misuses and pitfalls of P values and discuss potential mitigation strategies that leverage state-of-the-art statistical methods. The best practices always start with a sound study design including a robust data collection strategy to minimize data bias and a carefully thought-out analysis plan that can address potential misuses and pitfalls of P values. We highly encourage biomedical researchers to engage and involve statisticians from the very beginning of their studies.