Beyond URL Analysis: Evaluating Content-Based Phishing Detection Using HTML Structural Features

Abstract

For more than two decades, phishing pages have continued to evade defensive systems through increasingly sophisticated techniques. Most existing approaches combine URL features, telemetry, reputation signals, and HTML content, making it difficult to isolate the contribution of HTML alone. This study investigates whether phishing and legitimate pages can be distinguished using only HTML-derived content features. A balanced dataset of 40,000 pages was constructed through live HTTP retrieval, including 20,000 phishing pages from the verified PhishTank feed and 20,000 legitimate pages from the Tranco top-domain list. Using BeautifulSoup4, each page was represented by 15 HTML-derived features without using lexical URL features, reputation information, or network telemetry. Seven machine-learning classifiers were evaluated using 5-fold stratified cross-validation. Random Forest achieved the best performance with a positive-class F1 score of 0.940 ± 0.002 and a ROC-AUC of 0.983 ± 0.001, while producing a 6.5% false-negative rate and a 5.3% false-positive rate. Feature-importance analysis showed that n_links, n_scripts, n_images, and title_len accounted for approximately two-thirds of the overall importance ranking. An ablation study demonstrated that the five most important features achieved an F1 score of 0.926, 1.5 percentage points below the All-15 ablation result. SHAP analysis and error inspection revealed that structurally rich phishing pages can closely imitate legitimate websites. The findings demonstrate that static HTML structure provides substantial discriminative information for phishing detection while revealing important limitations of content-based approaches. No claims are made regarding temporal, cross-dataset, or adversarial generalization.