HTB AI Red Teamer — Module 2: when your model's blind spot is someone else's attack surface

Prakash’s learning notes, raw and honest, for future reference.

I just finished Module 2 of Hack The Box’s AI Red Teamer path. It’s called “Applications of AI in InfoSec,” and it’s essentially a crash course in building three classic security-focused ML models — spam classifier, network anomaly detector, malware image classifier — plus a skills assessment that threw a sentiment classifier at me.

I’m writing this down while it’s fresh, not as a tutorial, but as a record of what clicked, what broke, and the red teaming insights I’m already carrying into the next module. If you’re on the same path, or you’re a security analyst thinking about AI, I think you’ll find the red team angles at the end worth the read.

environment setup: conda, jupyter, and the MKL trap

#

I built everything on an Ubuntu 22.04 VM (8GB RAM, 4 cores, 80GB SSD). Miniconda for Python isolation, JupyterLab for the interactive coding surface, and a conda environment called ai with Python 3.11. The stack: numpy, scipy, pandas, scikit-learn, matplotlib, seaborn, plus the Hugging Face ecosystem (transformers, datasets, tokenizers, accelerate, evaluate), nltk, category_encoders, and PyTorch.

The first gotcha hit immediately. Installing PyTorch via conda gave me iJIT_NotifyEvent errors — an MKL conflict. Fix: uninstall the conda version and reinstall via pip with the CPU-only index. Then numpy 2.x broke scikit-learn and pandas, which were compiled against numpy 1.x. Fix: pip install "numpy<2". Classic dependency whack-a-mole.

VMware Tools (open-vm-tools) was necessary for copy-paste between the VM and my Windows host. I also ended up using scp to move files back and forth. A quick reminder: when you’re in a lab, the little convenience things matter. If you can’t paste a flag you just found, you’ll swear.

core concepts: what makes a good dataset, and why class imbalance is a security vulnerability

#

The module drills into data quality: relevance, completeness, consistency, representativeness, balance, size. But the real lesson is that class imbalance isn’t just a data science problem — it’s a security problem. If your training data has 77,000 normal samples and only 108 privilege escalation samples, your model will be blind to most privilege escalation attacks. An attacker who knows that can target that blind spot. We’ll see that brutally in the network anomaly model.

The preprocessing pipeline is: raw data → handle missing values → remove invalids → impute → encode categoricals → scale/transform → split. Tools: SimpleImputer (median for numeric, most_frequent for categories), OneHotEncoder, np.log1p() for skewed distributions, and train_test_split with a 60/20/20 split. Solid, repeatable, and the exact pipeline an attacker would study if they wanted to craft input that survives normalization but still fools the model.

Evaluation metrics I already knew from security work — accuracy, precision, recall, F1, confusion matrix — but seeing them from the attacker’s side reframes everything. Accuracy is meaningless if 99% of your traffic is normal. Precision matters if you’re drowning in false positives (alert fatigue). Recall matters if you want to catch the bad stuff. And for red teaming, the goal is to target recall: make your attack look like the negative class, and the model will miss it entirely. That’s the game.

model 1 — spam classifier (Naive Bayes)

#

• Algorithm: Multinomial Naive Bayes
• Dataset: UCI SMS Spam Collection (5,169 messages after dedup)
• Accuracy: ~91% on blind eval

Naive Bayes is dead simple: it calculates the probability that a given set of words belongs to spam vs. ham, assuming all words are independent. That assumption is naive, but for text it works surprisingly well. The classifier multiplies a bunch of conditional probabilities and picks the higher one.

Text preprocessing: lowercase, remove punctuation (but keep $ and ! — those are spam signals), tokenize, remove stop words, stem with PorterStemmer (so “running” becomes “run”, “entry” becomes “entri”), then rejoin into cleaned strings. Feature extraction used CountVectorizer with unigrams and bigrams, min_df=1, max_df=0.9. That turned 5,169 messages into a 37,069-feature matrix.

I tuned alpha (smoothing) with GridSearchCV, scoring on F1. Best alpha was 0.25. Blind evaluation gave ~91% accuracy. Not perfect, but not terrible.

Red teaming angle: Naive Bayes treats each word independently. That means inserting a bunch of legitimate words into a spam message shifts the probability toward ham. Stemming normalizes variants, so an attacker can craft words that stem differently than expected. And $/! carry spam weight — drop them, and the model gets less suspicious. The 9% that slips through is the adversarial surface. That’s the start of the evasion mindset.

model 2 — network anomaly detection (Random Forest)

#

• Algorithm: Random Forest classifier
• Dataset: NSL-KDD (148,517 network connections)
• Accuracy: ~99.76% on blind eval

Random Forest is an ensemble of decision trees. Each tree gets a bootstrap sample of the data and a random subset of features at each split. They vote, majority wins. The diversity reduces overfitting. Scikit-learn makes it a one-liner to fit, but the real work was in preprocessing.

NSL-KDD has 43 features per connection and five classes: normal, DoS, probe, privilege escalation, access. The class distribution was a wake-up call: 77,207 normal, 53,387 DoS (mostly Neptune), 14,077 probe, 3,738 access, and only 108 privilege escalation samples. That’s less than 0.07% of the data. I binarized the target for “attack or not” and also built a multiclass version.

After one-hot encoding protocol_type and service and joining with numeric features, I got 107 features. The Random Forest trained quickly and hit 99.76% on the blind eval for binary detection. But the per-class results told the real story:

Red teaming angle: This model catches 99% of DoS and probes, but misses 76% of privilege escalation attacks. The class imbalance meant the forest never learned what buffer_overflow, rootkit, or perl escalation look like. If I’m a red teamer targeting this model, I know exactly which attack class to use. This is the foundation of the evasion modules later: identify the model’s weak recall and walk through the gap. A real attacker doesn’t need to beat the whole model — just one poorly-represented class.

model 3 — malware image classifier (ResNet50 CNN)

#

• Algorithm: ResNet50 with transfer learning
• Dataset: Malimg (9,339 grayscale images, 25 malware families)
• Accuracy: 96.69% on blind eval

This was the mind-bender: take a malware binary, read each byte as a pixel intensity (0-255), render it as a grayscale image. Same malware family → similar visual texture. Then train a Convolutional Neural Network (CNN) to see those textures. No execution, no disassembly, just pictures.

I used ResNet50 pretrained on ImageNet. Froze all layers except the final fully-connected layer, replaced it with a 2048→1000→25 classifier. That meant I was training only about 2 million parameters instead of 23 million. Training took minutes per epoch, not hours. The model hit 97.55% on the training set and 96.69% on the blind eval.

PyTorch’s training loop is manual: zero gradients, forward pass, compute loss, backward pass, optimizer step. Very different from sklearn’s fit(). Saving the model required torch.jit.script, not joblib. The HTB evaluation server for this model expected a .pth file uploaded to port 8002.

Red teaming angle: CNNs are vulnerable to adversarial perturbations — tiny pixel changes invisible to humans that completely flip the classification. An attacker who knows the model architecture can craft a malware binary that, when rendered, looks like a different family’s byteplot. The foundation attacks (FGSM, DeepFool, JSMA) all exploit this. If you can get the model to misclassify your malware as a benign family, or as a family the analyst ignores, you win. This is what the evasion modules teach next.

model 4 — sentiment classifier (skills assessment)

#

• Algorithm: Multinomial Naive Bayes
• Dataset: IMDB movie reviews (50,000 reviews, balanced)
• Accuracy: 100% on blind eval

This one was a sprint. Same pipeline as Model 1: CountVectorizer with unigrams+bigrams, MultinomialNB, fit on the training set, predict on the test set. But the server side had quirks.

First attempt: I preprocessed the text (lowercase, removed punctuation, stemmed) before saving the model. The evaluation server sends raw text to predict(). My pipeline broke because the vectorizer expected already-cleaned tokens but got raw strings. Lesson: the pipeline must handle everything internally. External preprocessing = mismatch.

Second attempt: I used TfidfVectorizer + LogisticRegression, but the server returned null metrics — model format incompatible. Switched back to CountVectorizer + Naive Bayes with integer labels (0/1, not “positive”/“negative”), and it worked perfectly. 100% on the blind eval.

The whole episode was a reminder: when you’re deploying a model to someone else’s evaluation environment, the interface is pipeline.predict(raw_text). Every preprocessing step must live inside that pipeline.

the HTB eval server pattern

#

Every model followed the same dance:

1. Spawn target VM, get IP and port.
2. Connect HTB VPN (sudo openvpn academy-regular.ovpn).
3. Upload model from Jupyter using requests.post().

Simple, but the error messages were diagnostic tools themselves. Null metrics = format mismatch. 0% accuracy with a perfect-looking confusion matrix = label encoding mismatch (e.g., sending strings when the server expects integers). Learning to read those signals is half the battle.

red teaming insights I’m taking forward

#

This module was heavy on the building side, but every model came with a red teaming angle. Combined, they form a playbook for thinking like an adversarial ML attacker:

Know the algorithm = know the attack surface. Naive Bayes is sensitive to word frequency manipulation. Random Forest has blind spots where training data is thin. CNNs are vulnerable to pixel-level perturbations. The model’s architecture tells you how to break it.

Class imbalance is a security vulnerability, not a data quality footnote. The network anomaly model missed 76% of privilege escalation attacks because it had only 108 training samples. A real attacker who profiles the model will target those rare classes.

Preprocessing is attack surface. If you know a text classifier stems words and removes stop words, you can craft input that normalizes to benign tokens. The processing pipeline is part of the model’s logic, and it’s often less robust than the classifier itself.

Model evaluation servers leak information. Null metrics, format errors, specific response patterns — they tell you how the server calls your model. That’s recon.

Transfer learning = inherited biases. ResNet50 was trained on ImageNet (dogs, cats, furniture). Using it for malware byteplots works, but the model carries visual assumptions from natural images. Those assumptions might not align with security-relevant patterns, and an attacker can exploit the gap.

I’m not done with this path yet. Module 3 is where the offensive side really kicks in — evasion, adversarial examples, model stealing. But the foundation from Module 2 is solid: you can’t attack what you don’t understand, and you can’t defend against an attacker who knows your model’s blind spots better than you do.

If you’re a security analyst poking at AI, I’d say start here. Build the models first. See where they fail. Then flip the lens. That’s what I’m doing.

back to AI Fundamentals series index