The Internet runs faster and faster, the dangers in the digital world have increased and it
becomes more and more complicated to explore the web.
Once upon a time, when there was only Web 1.0 and the “good old” HTML, the biggest risk we ran was to find ourselves faced with a simple browser error. Who of us has never come across a “404 not found?”.
Now that, with Web 2.0, we interact and modify the contents of the pages directly online, we are exposed to daily threats that are undermined and hide behind simple actions such as clicking on a link.
Many have proposed and continue to propose solutions that we can use to protect ourselves and defend ourselves against cyber-attacks. In this article we will see ntopng, one of the best known solutions especially in the open source world.
Presentation
Ntopng was born as a traffic analysis tool and over time it has “evolved” to become an application filter. The project’s author is Prof. Luca Deri, “Research Scientist and Network Manager” at the Department of Computer Science of the University of Pisa.
Visiting the web page of Luca, http://luca.ntop.org/, we can find all his publications made over the years, get an idea about the scope of his scientific research and understand a little ‘what the spirit with which he created ntopng.
In this regard it is a must to quote one of his sentences: “The Internet today represents me free radio represented in the 70s. I think, do, create. Herein you can find all of the years that is my humble and tiny contribution to computer science (r) evolution.“
What is NtopNG
ntopng is a traffic analysis networking tool that offers unprecedented visibility on packets traveling on the network.
One of the most interesting features of the latest version of ntopng is undoubtedly that of application filter, thanks to which we can control more than 250 applications including Facebook, Youtube, WhatsApp, Skype and Tor, blocking or limiting the bandwidth of requests client and preventing, in fact, their uncontrolled use. Now let’s look at some functions and discover their potential.
Overview of features
ntopng is released in three different versions: Community, Professional and Enterprise. The various features are shown in the following comparative table.JTVCYWRyb3RhdGUlMjBiYW5uZXIlM0QlMjIzJTIyJTVEIntroduction
The Internet runs faster and faster, the dangers in the digital world have increased and it
becomes more and more complicated to explore the web.
Once upon a time, when there was only Web 1.0 and the “good old” HTML, the biggest risk we ran was to find ourselves faced with a simple browser error. Who of us has never come across a “404 not found?”.
Now that, with Web 2.0, we interact and modify the contents of the pages directly online, we are exposed to daily threats that are undermined and hide behind simple actions such as clicking on a link.
Many have proposed and continue to propose solutions that we can use to protect ourselves and defend ourselves against cyber-attacks. In this article we will see ntopng, one of the best known solutions especially in the open source world.
Presentation
Ntopng was born as a traffic analysis tool and over time it has “evolved” to become an application filter. The project’s author is Prof. Luca Deri, “Research Scientist and Network Manager” at the Department of Computer Science of the University of Pisa.
Visiting the web page of Luca, http://luca.ntop.org/, we can find all his publications made over the years, get an idea about the scope of his scientific research and understand a little ‘what the spirit with which he created ntopng.
In this regard it is a must to quote one of his sentences: “The Internet today represents me free radio represented in the 70s. I think, do, create. Herein you can find all of the years that is my humble and tiny contribution to computer science (r) evolution.“
What is NtopNG
ntopng is a traffic analysis networking tool that offers unprecedented visibility on packets traveling on the network.
One of the most interesting features of the latest version of ntopng is undoubtedly that of application filter, thanks to which we can control more than 250 applications including Facebook, Youtube, WhatsApp, Skype and Tor, blocking or limiting the bandwidth of requests client and preventing, in fact, their uncontrolled use. Now let’s look at some functions and discover their potential.
Overview of features
ntopng is released in three different versions: Community, Professional and Enterprise. The various features are shown in the following comparative table.Generation of alarms based on time/traffic thresholds or suspicious behavior such as visiting a malicious site
| Feature | Community | Professional | Enterprise |
|---|---|---|---|
| Monitoring of active flows and hosts of the network † | ✓ | ✓ | ✓ |
| Identification of application protocols (Facebook, Youtube, BitTorrent, etc) in traffic | ✓ | ✓ | ✓ |
| Recording and display of the use of application protocols for each host over time | ✓ | ✓ | ✓ |
| Grouping of hosts for VLAN, Operating System, Country, and Autonomous Systems | ✓ | ✓ | ✓ |
| Geographical map of network communications made by each host | ✓ | ✓ | ✓ |
| Identification of the top talker hosts (senders and receivers) with resolution per minute | ✓ | ✓ | ✓ |
| View the most requested HTTP sites from each host | ✓ | ✓ | ✓ |
| Export of communications on MySQL and ElasticSearch | ✓ | ✓ | ✓ |
| Generation of alarms based on time / traffic thresholds or suspicious behavior such as visiting a malicious site | ✓ | ✓ | ✓ |
| Alarms and warnings such as Slack messages | ✓ | ✓ | ✓ |
| Display of traffic for each VLAN | ✓ | ✓ | ✓ |
| Data collecting from nProbe to process the remote interfaces monitored by nProbe and flow export devices (eg routers and switches) as if they were local | ✓ | ✓ | ✓ |
| Displaying data collected by nProbe | ✓ | ✓ | ✓ |
| Grouping hosts into logical sets of IP and MAC addresses known as hosts pools †† | ✓ | ✓ | ✓ |
| Real-time view of the top talkers and application protocols and comparison with daily activities | ✗ | ✓ | ✓ |
| Browsing the registered MySQL data to identify the cause of network problems | ✗ | ✓ | ✓ |
| Generation of graphical reports with the top hosts, application protocols, countries, networks and autonomous systems in configurable time periods | ✗ | ✓ | ✓ |
| Traffic history based on profiles created using BPF (Berkeley Packet Filter) syntax ‡ | ✗ | ✓ | ✓ |
| Limiting/blocking host traffic with custom policies for each protocol * | ✗ | ✓ | ✓ |
| Integration with LDAP authentication servers | ✗ | ✓ | ✓ |
| Query SNMP devices for data such as port status, traffic and MAC address information | ✗ | ✓ | ✓ |
| Integration with Nagios * | ✗ | ✓ | ✓ |
| MySQL insertions to get writes to the fastest 5x database | ✗ | ✗ | ✓ |
| Data aggregation in MySQL for faster historical explorations | ✗ | ✗ | ✓ |
| Generate traffic and total activity reports for any host, network or interface | ✗ | ✗ | ✓ |
| Detection of attackers and victims through real-time alerts | ✗ | ✗ | ✓ |
| Exploration and filtering of alarms | ✗ | ✗ | ✓ |
| Viewing and storing traffic by SNMP port | ✗ | ✗ | ✓ |
| Viewing and storing NetFlow/sFlow device data | ✗ | ✗ | ✓ |
| Captive Portal for Internet browsing * | ✗ | ✗ | ✓ |
| Daily traffic quotas that are applied to clients * | ✗ | ✗ | ✓ |
| Parental control with the DNS integration of SafeSearch * | ✗ | ✗ | ✓ |
* Feature not available with Windows
† The Enterprise version allows simultaneous monitoring of up to 128 different network interfaces. Professional and Community versions allow monitoring of up to 32 different interfaces.
†† The Enterprise version allows simultaneous monitoring of up to 128 different host pools. Professional and Community versions allow simultaneous monitoring of up to 3 different host pools.
‡ The Enterprise version allows simultaneous monitoring of up to 128 different traffic profiles. The Professional version allows the creation of 16 traffic profiles.
Supported platforms**
- Unix (including Linux, * BSD, and MacOSX)
- Windows x64 (including the latest Windows 10)
- ARM
Web GUI
- Available via HTML5-ready/li web browser
- SSL / HTTPS support
Requirements
- Memory usage Depends on the ntop configuration, the number of hosts, and the number of active TCP sessions. Generally it varies from a few MB (small LAN) to 100 MB for a WAN.
- Use of CPUD depends on the ntop configuration and traffic conditions. On a modern PC and on a large LAN, it is less than 10% of the total CPU load.
Protocols
- Ethernet
- IPv4/IPv6
- TCP/UDP/ICMP
- GRE
- DHCP/BOOTP/NetBIOS/DNS…
- 250+ applications with Layer-7 protocol supported with nDPI
- …many others.
Extensibility
- Compatibility with scripts in LUA
- Web interface extensions without having to change the Ntopng C ++ engine.
Additional features
- sFlow, NetFlow (including v5 and v9) and IPFIX supported via nProbe (collection from multiple nProbes is supported).
- Statistic for: Internet Domain, AS (Autonomous Systems), VLAN (Virtual LAN).
- Decoding protocol for all application protocols supported by nDPI.
**NtopNg is also available for Ubiquiti EdgeRouter (Lite or X).
Installation
Installation is very simple. After installing your preferred operating system, simply download the packages and install them following the instructions on http://packages.ntop.org.
LAB
We have done two laboratories, one of only monitoring and the other with filtering (you need a license – see table of characteristics). The latter also includes the first one, so if you have already purchased a license, you can skip reading the monitoring lab.
Monitoring Lab
In this lab we used the Compact Small UTM appliance as a tool to analyze our network traffic.
To make it possible to have a copy of the traffic to be analyzed, we put the affected interface of our firewall A1 Server Alluminium in SPAN: for us it was the LAN interface. The same thing can also be done on a switch that supports the configuration of the SPAN.
Once entered in ntopng, in the navigation menu we find, at the top, the voice Interface. By clicking on it, we can see which is the NIC that is physically connected to the firewall.[vc_single_image image=”13677″ img_size=”full” alignment=”center” css=”.vc_custom_1521559475830{margin-top: -20px !important;padding-top: 10px !important;padding-right: 10px !important;padding-bottom: 10px !important;padding-left: 10px !important;background-color: #f4f4f2 !important;}”]Below is the network scheme that best represents what is written:[vc_single_image image=”13680″ img_size=”full” alignment=”center” css=”.vc_custom_1521559783048{margin-top: -20px !important;padding-top: 10px !important;padding-right: 10px !important;padding-bottom: 10px !important;padding-left: 10px !important;background-color: #f4f4f2 !important;}”]As shown in the diagram, ntopng is connected “in parallel” to the firewall and is therefore “invisible”.
LAN interface traffic is duplicated on the new SPAN interface. The latter is the INPUT interface of ntopng.
In this first configuration mode, we analyzed traffic for about two days to better understand the type of traffic on the network. This modality, however, has the sole purpose of analysis and is therefore “passive”. From here, we can not limit or block anything. Given the potential of the instrument, we chose almost immediately to reconfigure it with a bridge.
Application Filtering Lab
Also in this lab we used the Compact Small UTM UTM appliance with ntopng as a tool for analyzing our network traffic and for filtering.
To make sure that it could be in line with our A1 Server Alluminium we have created a bridge between two ntopng NICs.
So in this second lab we wanted to test the filtering. This function, really useful, is certainly interesting for those who, like us, do not confine the problem of security in the dark corner of the company.
Bridge: the traffic of our LAN, which goes to the uplink on the internet, instead of going to the firewall is then hijacked in ntopng. The latter has been configured with two NICs in bridge [note: two NICs of ntopng must be used for each bridge] so it is able to analyze the traffic and “pass” only “legitimate” packets to the firewall. Once you have entered Ntopng, in the interface menu you will find the bridge you have created.[vc_single_image image=”13683″ img_size=”full” alignment=”center” css=”.vc_custom_1521560333376{margin-top: -20px !important;padding-top: 10px !important;padding-right: 10px !important;padding-bottom: 10px !important;padding-left: 10px !important;background-color: #f4f4f2 !important;}”]Below is the network scheme that best represents what is written:[vc_single_image image=”13683″ img_size=”full” alignment=”center” css=”.vc_custom_1521560493758{margin-top: -20px !important;padding-top: 10px !important;padding-right: 10px !important;padding-bottom: 10px !important;padding-left: 10px !important;background-color: #f4f4f2 !important;}”]In our lab (see image below), we have made a very simple 2Mb/s limiter (traffic shaper) and put it in some policies. In others, however, as for the family of social networks, we have chosen not to allow the transit of packages.[vc_single_image image=”13685″ img_size=”full” alignment=”center” css=”.vc_custom_1521560728136{margin-top: -20px !important;padding-top: 10px !important;padding-right: 10px !important;padding-bottom: 10px !important;padding-left: 10px !important;background-color: #f4f4f2 !important;}”]All in a very intuitive, simple and fast.
Below is an image showing how to create the bridge in the NtopNG configuration file:[vc_single_image image=”13687″ img_size=”full” alignment=”center” css=”.vc_custom_1521561139164{margin-top: -20px !important;padding-top: 10px !important;padding-right: 10px !important;padding-bottom: 10px !important;padding-left: 10px !important;background-color: #f4f4f2 !important;}”]Conclusions
Perhaps the description of the labs is not immediately evident, but application filtering obviously implies monitoring.
This means that we can continue to monitor our network even if we configure the appliance in “active” mode as shown previously.
ntopng is, therefore, indispensable for anyone who does not want to be a passive viewer of the traffic generated on his company network.
It is also very useful for all those who, for enforcement, or for simple need – think for example to those who do not have sufficient connectivity to support the volume of data today – must have traffic control. Now you know how to do it.
