At home I use a router with OPNsense. It's a very advanced router and one of the possibilities is sending netflow data. Till now I had nothing to do with the data, beside the simple interface in OPNsense. In a far history, I played with the old ntop, but that is really outdated. But luckely there is ntop-ng. It's open source, but they sell some extra features. Not really the software I would look for, but it does a good job! So I give it a try.
The software installation wasn't bad, but there are some strange bug's at the time of writing. I did this on a clean Debian server, version Jessie.
I did almost the same as on http://packages.ntop.org/apt-stable/ but only left out nbox, it didn't exsist in the repo, so it's not installable...
wget http://apt-stable.ntop.org/jessie/all/apt-ntop-stable.deb
dpkg -i apt-ntop-stable.deb
apt-get clean all
apt-get update
apt-get install pfring nprobe ntopng ntopng-data n2disk cento
First, stop everything. In default mode it start collecting data directly from the networkinterface. But we only want to parse netflow data, so we don't need the interface information. And I had to kill the process. No idea why, but the stop script won't stop the service...
service ntopng stop
killall ntopng
Then I had to flush the redis database. All the settings made in the interface will be gone! Keep that in mind if you do this!
root@ntop:~# redis-cli
127.0.0.1:6379> FLUSHALL
OK
Now we can start with the real configuration. I got this from https://blog.webernetz.net/2016/08/16/using-netflow-with-nprobe-for-ntopng/
First we configure the nprobe, a converter from netflow to Zeromq data. Make a new file /etc/nprobe/nprobe-none.conf
--zmq="tcp://127.0.0.1:5556"
--collector-port=2055
-n=none
-i=none
Make an empty startfile. In my opinion a very strange way of telling a service to start. It's almost undocumented en needed for both nprobe and ntopng.
touch /etc/nprobe/nprobe-none.start
Now let's configure ntopng. Change in /etc/ntopng/ntopng.conf:
--interface="tcp://127.0.0.1:5556"
--http-port=3000
-m 10.0.0/24,10.0.1.0/24
--community
This will start the ZeroMQ and webinterface port. It defines all you internal subnets. I got a lot of networks and remote VPN's, so this could become a long list, but really good to define! Use the '-m' and not the long '--local-networks' version. In my version the long one didn't work. The manpage had two long options for '-m', but in the last version I couldn't find it anymore. Probably it's fixed.
And I use the community option. Otherwise you are stuk with 10 minutes of using the pro version before switching back to the open source version, every time after rebooting... There are a ton of other options to set. You can find them all in the manpage.
And again, make an empty startfile.
touch /etc/ntopng/ntopng.start
Start everything with:
service nprobe start
service ntopng start
See your new collector at http://server_ip:3000/ Default login/password is admin/admin. Send your netflow data to server_ip:2055. Happy collecting!
You can use Mysql as a backend server. It's not for all the data, but is't a good place to query stuff. Also the long term graphs are working in the interface. First we need a mysql server.
apt-get install mysql-server
Configure your mysql user to send the data.
root@ntop:~# mysql -p
Enter password:
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 45
Server version: 5.5.52-0+deb8u1 (Debian)
Copyright (c) 2000, 2016, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> create database ntopng;
Query OK, 1 row affected (0.00 sec)
mysql> CREATE USER 'ntopng'@'localhost' IDENTIFIED BY 'Yoursecretpassword';
Query OK, 0 rows affected (0.00 sec)
mysql> GRANT ALL PRIVILEGES ON ntopng . * TO 'ntopng'@'localhost';
Query OK, 0 rows affected (0.00 sec)
And tell ntop to send data to the database. Add to /etc/ntopng/ntopng.conf:
-F "mysql;<server>;<database>;<table>;<user>;<password>"
and restart ntopng.
If you want your website working behind a proxy with the use of an extra contex root, like apache of hiawatha, there is a flaw in de man page. From the manpage:
-Z|--http-prefix <prefix>
HTTP prefix to be prepended to URLs. This is useful when using ntopng behind
a proxy. E.g. if you want to make the ntopng web interface accessible
through a proxy at a certain IP address with the /ntopng/ base URL and you
have the following lines in your proxy's configuration:
ProxyPass /ntopng/ http://192.168.0.3:3000/ntopng/
ProxyPassReverse /ntopng/ http://192.168.0.3:3000/ntopng/
you must use ntopng with -Z "/ntopng/"
But this is acqually wrong. At least, it didn't work out for me. The last extra slash is not working. So I added to /etc/ntopng/ntopng.conf:
-Z "/ntopng"
And that did the job.
ntop-ng is a very nice way of visualising you netflow data. It still needs a lot of work and not everything is working out of the box, but it has a lot of potential! You can even make a connection to your elasticsearch cluster to collect the parsed data. Nice options, but keep in mind that there are a lot of bugs!