[CB16] Around the Web in 80 Hours: Scalable Fingerprinting with Chromium Automationby Isaac Dawson

Building a distributed scanner can be challenging, building one using real browsers even more so.
Injecting JavaScript to extract JS libraries and their versions, storing all HTML and JavaScript along with security headers requires a unique architecture. Having scanned the top 1,000,000 sites, I will cover the challenges I overcame in designing a scalable system to fingerprint the current state of the web. I will also present some of the more interesting findings of the data that was analyzed.

Isaac Dawson
Isaac Dawson is a Principal Security Researcher at Veracode, Inc. where he leads the R&D efforts of Veracode’s dynamic analysis offerings. Prior to Veracode, he was a consultant for @stake and then Symantec. In 2004 he moved to Japan to start their application security consulting team.
After leaving for Veracode, he decided Japan was just too comfortable and has stayed ever since.
An avid go programmer, he has an interest in distributed systems and in particular, scanning the web.

http://codeblue.jp/2016/en/contents/speakers.html#speaker-dawson