URL Crawling & Classification System.pdf

(2216 KB) Pobierz

URL Crawling & classification system

Emil Lindgjerdet Vaagland

Master of Science in Communication Technology

Submission date: June 2012

Supervisor:

Svein Johan Knapskog, ITEM

Norwegian University of Science and Technology

Department of Telematics

Problem Description

Name of student: Emil Vaagland

Malware is commonly hosted on hacked websites or temporary malicious servers. The

two-fold goal of this project is to:

1. Create a system to harvest potentially malicious URLs by crawling the web.

2. Create a system to give each URL a reputation-score and classify it as malicious

or not without blacklisting clean sites.

Assignment given: 16.01.2012

Supervisor: Svein Johan Knapskog (ITEM, NTNU)

Co-supervisors: Felix Leder (Norman ASA), Trygve Brox (Norman ASA)

URL CRAWLING AND CLASSIFICATION SYSTEM

Emil Vaagland

June 2012