A major overhaul of threat intelligence in Red Hand
Threat intelligence v3 replaces a dominant aggregated feed with about 145 original sources, preserving source context while removing stale and shared-infrastructure indicators.
We just pushed a fairly major overhaul of threat intelligence in Red Hand.
I should probably start with a disclaimer: I don't think threat feeds are nearly as important as parts of the industry make them sound. A depressing amount of IP reputation data is garbage, and against a targeted attacker you should assume IPs, domains and binaries are disposable anyway. Threat intel is mostly useful for low-hanging fruit, reused infrastructure, commodity malware, scanners, phishing and botnets. Useful, yes. Magic, no.
That was also part of the reason for rebuilding ours.
With v3, the goal was to stop depending on aggregators and get as close to the original sources as possible. Aggregation sounds convenient, but the more feeds get mixed together, the more context disappears. Malicious and suspicious get flattened into the same label, timestamps become questionable, and you lose who actually observed the indicator, where it came from and what activity it represented.
Eventually you end up with a very large list of things the internet once disliked.
We had one aggregated feed like that with roughly 5 million IPs, making up about 86% of our previous IP intelligence. Much of it was stale, duplicated or impossible to properly qualify. We removed it, traced the data back to the original publishers, split that feed into more than 30 separate sources, and checked what each one actually contributes.
The new build now uses about 145 original sources, covering roughly 1.5M IPs and 23M domains. One number I like more than the totals is what we throw away. We now resolve malicious domains ourselves every night, producing roughly 850k IPs, but about 32% are discarded because they're shared infrastructure such as CDNs, cloud front-ends, public resolvers, shared hosting, parking pages and sinkholes. Keeping them would make the database bigger and the detections worse.
The data now keeps its original source and context, stale indicators disappear much faster, every feed is health-checked, and everything is integrated directly into Red Hand's PCAP and live traffic analysis.
So v3 contains far fewer IPs than the previous version, while giving us better coverage and much better context. That's a trade I'll take every time.