Skip to content

Commit b5e8019

Browse files
jorritfolmerjorritfolmer
committed
Added test for Fortigate FortiOS 5.2
1 parent c332b60 commit b5e8019

8 files changed

+103
-126
lines changed

‎CHANGELOG.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,8 @@
1+
## 3.5.2
2+
3+
- Added test for Fortigate FortiOS 5.2 (Netflow v9)
4+
- Clarified confusing warning about missing templates
5+
16
## 3.5.1
27

38
- Added test for Barracuda firewall (IPFIX)

‎CONTRIBUTORS

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@ Contributors:
99
* Diyaldine Maoulida
1010
* Evgeniy Sudyr (ejectck)
1111
* G.J. Moed (gjmoed)
12+
* GMoz
1213
* Jordan Sissel (jordansissel)
1314
* Jorrit Folmer (jorritfolmer)
1415
* Keenan Tims (ktims)

‎docs/index.asciidoc

Lines changed: 26 additions & 36 deletions
Original file line numberDiff line numberDiff line change
@@ -24,56 +24,46 @@ The "netflow" codec is used for decoding Netflow v5/v9/v10 (IPFIX) flows.
2424

2525
==== Supported Netflow/IPFIX exporters
2626

27+
This codec supports:
28+
29+
* Netflow v5
30+
* Netflow v9
31+
* IPFIX
32+
2733
The following Netflow/IPFIX exporters are known to work with the most recent version of the netflow codec:
2834

2935
[cols="6,^2,^2,^2,12",options="header"]
3036
|===========================================================================================
31-
|Netflow exporter | v5 | v9 | IPFIX | Remarks
32-
|Softflowd | y | y | y | IPFIX supported in http://github.com/djmdjm/softflowd
33-
|nProbe | y | y | y |
34-
|ipt_NETFLOW | y | y | y |
35-
|Cisco ASA | | y | |
36-
|Cisco IOS 12.x | | y | |
37-
|fprobe | y | | |
38-
|Juniper MX80 | y | | | SW > 12.3R8
39-
|OpenBSD pflow | y | n | y | http://man.openbsd.org/OpenBSD-current/man4/pflow.4
40-
|Mikrotik 6.35.4 | y | | n | http://wiki.mikrotik.com/wiki/Manual:IP/Traffic_Flow
41-
|Ubiquiti Edgerouter X | | y | | With MPLS labels
42-
|Citrix Netscaler | | | y | Still some unknown fields, labeled netscalerUnknown<id>
37+
|Netflow exporter | v5 | v9 | IPFIX | Remarks
38+
|Barracuda Firewall | | | y |
39+
|Cisco ASA | | y | |
40+
|Cisco ASR | | y | |
41+
|Cisco IOS 12.x | | y | |
42+
|Cisco WLC | | y | |
43+
|Citrix Netscaler | | | y | Still some unknown fields, labeled netscalerUnknown<id>
44+
|fprobe | y | | |
45+
|Fortigate FortiOS 5.2 | | y | |
46+
|ipt_NETFLOW | y | y | y |
47+
|Juniper MX80 | y | | | SW > 12.3R8
48+
|Mikrotik 6.35.4 | y | | n | http://wiki.mikrotik.com/wiki/Manual:IP/Traffic_Flow
49+
|nProbe | y | y | y |
50+
|OpenBSD pflow | y | n | y | http://man.openbsd.org/OpenBSD-current/man4/pflow.4
51+
|Softflowd | y | y | y | IPFIX supported in http://github.com/djmdjm/softflowd
52+
|Streamcore Streamgroomer | | y | |
53+
|Ubiquiti Edgerouter X | | y | | With MPLS labels
54+
|VMware VDS | | | y | Still some unknown fields
4355
|===========================================================================================
4456

4557
==== Usage
4658

47-
Example Logstash configuration:
59+
Example Logstash configuration that will listen on 2055/udp for Netflow v5,v9 and IPFIX:
4860

4961
[source, ruby]
5062
--------------------------
5163
input {
5264
udp {
53-
host => localhost
5465
port => 2055
55-
codec => netflow {
56-
versions => [5, 9]
57-
}
58-
type => netflow
59-
}
60-
udp {
61-
host => localhost
62-
port => 4739
63-
codec => netflow {
64-
versions => [10]
65-
target => ipfix
66-
}
67-
type => ipfix
68-
}
69-
tcp {
70-
host => localhost
71-
port => 4739
72-
codec => netflow {
73-
versions => [10]
74-
target => ipfix
75-
}
76-
type => ipfix
66+
codec => netflow
7767
}
7868
}
7969
--------------------------

‎lib/logstash/codecs/netflow.rb

Lines changed: 1 addition & 90 deletions
Original file line numberDiff line numberDiff line change
@@ -5,63 +5,7 @@
55
#require "logstash/json"
66
require "json"
77

8-
# The "netflow" codec is used for decoding Netflow v5/v9/v10 (IPFIX) flows.
9-
#
10-
# ==== Supported Netflow/IPFIX exporters
11-
#
12-
# The following Netflow/IPFIX exporters are known to work with the most recent version of the netflow codec:
13-
#
14-
# [cols="6,^2,^2,^2,12",options="header"]
15-
# |===========================================================================================
16-
# |Netflow exporter | v5 | v9 | IPFIX | Remarks
17-
# |Softflowd | y | y | y | IPFIX supported in http://github.com/djmdjm/softflowd
18-
# |nProbe | y | y | y |
19-
# |ipt_NETFLOW | y | y | y |
20-
# |Cisco ASA | | y | |
21-
# |Cisco IOS 12.x | | y | |
22-
# |fprobe | y | | |
23-
# |Juniper MX80 | y | | | SW > 12.3R8
24-
# |OpenBSD pflow | y | n | y | http://man.openbsd.org/OpenBSD-current/man4/pflow.4
25-
# |Mikrotik 6.35.4 | y | | n | http://wiki.mikrotik.com/wiki/Manual:IP/Traffic_Flow
26-
# |Ubiquiti Edgerouter X | | y | | With MPLS labels
27-
# |Citrix Netscaler | | | y | Still some unknown fields, labeled netscalerUnknown<id>
28-
# |===========================================================================================
29-
#
30-
# ==== Usage
31-
#
32-
# Example Logstash configuration:
33-
#
34-
# [source, ruby]
35-
# --------------------------
36-
# input {
37-
# udp {
38-
# host => localhost
39-
# port => 2055
40-
# codec => netflow {
41-
# versions => [5, 9]
42-
# }
43-
# type => netflow
44-
# }
45-
# udp {
46-
# host => localhost
47-
# port => 4739
48-
# codec => netflow {
49-
# versions => [10]
50-
# target => ipfix
51-
# }
52-
# type => ipfix
53-
# }
54-
# tcp {
55-
# host => localhost
56-
# port => 4739
57-
# codec => netflow {
58-
# versions => [10]
59-
# target => ipfix
60-
# }
61-
# type => ipfix
62-
# }
63-
# }
64-
# --------------------------
8+
# Documentation moved to docs/
659

6610
class LogStash::Codecs::Netflow < LogStash::Codecs::Base
6711
config_name "netflow"
@@ -87,42 +31,9 @@ class LogStash::Codecs::Netflow < LogStash::Codecs::Base
8731
config :versions, :validate => :array, :default => [5, 9, 10]
8832

8933
# Override YAML file containing Netflow field definitions
90-
#
91-
# Each Netflow field is defined like so:
92-
#
93-
# [source,yaml]
94-
# --------------------------
95-
# id:
96-
# - default length in bytes
97-
# - :name
98-
# id:
99-
# - :uintN or :ip4_addr or :ip6_addr or :mac_addr or :string
100-
# - :name
101-
# id:
102-
# - :skip
103-
# --------------------------
104-
#
105-
# See <http://github.com/logstash-plugins/logstash-codec-netflow/blob/master/lib/logstash/codecs/netflow/netflow.yaml> for the base set.
10634
config :netflow_definitions, :validate => :path
10735

10836
# Override YAML file containing IPFIX field definitions
109-
#
110-
# Very similar to the Netflow version except there is a top level Private
111-
# Enterprise Number (PEN) key added:
112-
#
113-
# [source,yaml]
114-
# --------------------------
115-
# pen:
116-
# id:
117-
# - :uintN or :ip4_addr or :ip6_addr or :mac_addr or :string
118-
# - :name
119-
# id:
120-
# - :skip
121-
# --------------------------
122-
#
123-
# There is an implicit PEN 0 for the standard fields.
124-
#
125-
# See <http://github.com/logstash-plugins/logstash-codec-netflow/blob/master/lib/logstash/codecs/netflow/ipfix.yaml> for the base set.
12637
config :ipfix_definitions, :validate => :path
12738

12839
NETFLOW5_FIELDS = ['version', 'flow_seq_num', 'engine_type', 'engine_id', 'sampling_algorithm', 'sampling_interval', 'flow_records']

‎spec/codecs/netflow9_test_fortigate_fortios_521_data256.dat

60 Bytes
Binary file not shown.

‎spec/codecs/netflow9_test_fortigate_fortios_521_data257.dat

76 Bytes
Binary file not shown.

‎spec/codecs/netflow9_test_fortigate_fortios_521_tpl.dat

284 Bytes
Binary file not shown.

‎spec/codecs/netflow_spec.rb

Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -966,6 +966,76 @@
966966

967967
end
968968

969+
context "Netflow 9 Fortigate FortiOS 5.2.1" do
970+
let(:data) do
971+
packets = []
972+
packets << IO.read(File.join(File.dirname(__FILE__), "netflow9_test_fortigate_fortios_521_tpl.dat"), :mode => "rb")
973+
packets << IO.read(File.join(File.dirname(__FILE__), "netflow9_test_fortigate_fortios_521_data256.dat"), :mode => "rb")
974+
packets << IO.read(File.join(File.dirname(__FILE__), "netflow9_test_fortigate_fortios_521_data257.dat"), :mode => "rb")
975+
end
976+
977+
let(:json_events) do
978+
events = []
979+
events << <<-END
980+
{
981+
"netflow": {
982+
"flow_seq_num": 13641,
983+
"scope_system": 1,
984+
"total_bytes_exp": 6871319015,
985+
"total_flows_exp": 107864,
986+
"flow_active_timeout": 1800,
987+
"flow_inactive_timeout": 15,
988+
"flowset_id": 256,
989+
"total_pkts_exp": 11920854,
990+
"version": 9,
991+
"sampling_algorithm": 1,
992+
"sampling_interval": 1
993+
},
994+
"@timestamp": "2017-07-18T05:42:14.000Z",
995+
"@version": "1"
996+
}
997+
END
998+
999+
events << <<-END
1000+
{
1001+
"netflow": {
1002+
"output_snmp": 3,
1003+
"in_pkts": 3,
1004+
"ipv4_dst_addr": "31.13.87.36",
1005+
"first_switched": "2017-07-25T04:44:29.999Z",
1006+
"flowset_id": 257,
1007+
"l4_src_port": 61910,
1008+
"version": 9,
1009+
"flow_seq_num": 13635,
1010+
"ipv4_src_addr": "192.168.99.7",
1011+
"in_bytes": 152,
1012+
"protocol": 6,
1013+
"last_switched": "2017-07-25T04:44:38.999Z",
1014+
"input_snmp": 9,
1015+
"out_pkts": 0,
1016+
"out_bytes": 0,
1017+
"l4_dst_port": 443
1018+
},
1019+
"@timestamp": "2017-07-18T05:41:59.000Z",
1020+
"@version": "1"
1021+
}
1022+
END
1023+
events.map{|event| event.gsub(/\s+/, "")}
1024+
end
1025+
1026+
it "should decode raw data" do
1027+
expect(decode.size).to eq(2)
1028+
expect(decode[0].get("[netflow][total_bytes_exp]")).to eq(6871319015)
1029+
expect(decode[1].get("[netflow][ipv4_src_addr]")).to eq("192.168.99.7")
1030+
end
1031+
1032+
it "should serialize to json" do
1033+
expect(JSON.parse(decode[0].to_json)).to eq(JSON.parse(json_events[0]))
1034+
expect(JSON.parse(decode[1].to_json)).to eq(JSON.parse(json_events[1]))
1035+
end
1036+
1037+
end
1038+
9691039
context "Netflow 9 Streamcore" do
9701040
let(:data) do
9711041
packets = []

0 commit comments

Comments
 (0)