Skip site navigation (1)Skip section navigation (2)
Date:      Sun, 2 Mar 2014 21:21:17 -0500
From:      Chad Gross <avatar4d@gmail.com>
To:        freebsd-ports@freebsd.org
Cc:        5u623l20@gmail.com, Alex Samorukov <samm@os2.kiev.ua>
Subject:   Re: [patch] net-mgmt/flowviewer and security/silktools patches
Message-ID:  <CAHP1p-VPyvEShTJZcpfy_iS3SRWwdwhZ6wUfS15N5z1HBBKmKw@mail.gmail.com>
In-Reply-To: <CAHP1p-UDykoxtVpuTq6gMPw3AGNe0kgsgod9wqWee4zEE29pKA@mail.gmail.com>
References:  <CAHP1p-Xq_Kct7=U3nXsPO_ariQZ7x=vc3ybXj7ekMjmG_iR4uA@mail.gmail.com> <CAHP1p-UDykoxtVpuTq6gMPw3AGNe0kgsgod9wqWee4zEE29pKA@mail.gmail.com>

next in thread | previous in thread | raw e-mail | index | archive | help
On Tue, Feb 18, 2014 at 1:57 PM, Chad Gross <avatar4d@gmail.com> wrote:

> On Tue, Feb 18, 2014 at 10:33 AM, Chad Gross <avatar4d@gmail.com> wrote:
>
>> I managed to configure net-mgmt/flowviewer with security/silktools, but
>> had to make some modifications to get it working. FlowViewer is configured
>> by defaut to pass the $silk_data_dir + $device_name as the root data
>> directory to the rwfilter tool, when the root directory should be the same
>> as $silk_data_dir. I've confirmed it is still the configured this way in
>> the latest version (4.3, released 2/11/14) so I could be misconfiguring
>> something, but I don't see how since I following the documentation (
>> http://sourceforge.net/projects/flowviewer/files/FlowViewer.pdf/download).
>> I also manually ran the commands out of working/DEBUG_VIEWER and it
>> produced nothing until I updated --data-rootdir=/data/flows/S0 to
>>  --data-rootdir=/data/flows.
>>
>> Here are patches for the 4 affected files:
>>
>>
>> --- FlowGrapher_Main.cgi.orig   2014-02-18 08:49:42.000000000 -0500
>>
>> +++ FlowGrapher_Main.cgi        2014-02-18 09:09:58.000000000 -0500
>>
>> @@ -535,7 +535,7 @@
>>
>>                 $silk_flow_type =~ s/\s+//g;
>>
>>         }
>>
>>
>>
>> -       $data_root_dir = $silk_data_directory ."/". $device_name;
>>
>> +       $data_root_dir = $silk_data_directory;
>>
>>
>>
>>         # Prepare rwfilter start and end time parameters, filter criteria
>> and window type
>>
>>
>> --- FlowTracker_Recreate.orig   2014-02-16 15:50:35.000000000 -0500
>>
>> +++ FlowTracker_Recreate        2014-02-18 09:09:58.000000000 -0500
>>
>> @@ -245,7 +245,7 @@
>>
>>                         $cat_start =
>> epoch_to_date($cat_start_epoch,"LOCAL");
>>
>>                         $cat_end   =
>> epoch_to_date($cat_end_epoch,"LOCAL");
>>
>>
>>
>> -                       $data_root_dir = $silk_data_directory ."/".
>> $device_name;
>>
>> +                       $data_root_dir = $silk_data_directory;
>>
>>
>>
>>                         $silk_flow_type = "";
>>
>>
>>
>> --- FlowTracker_Collector.orig  2014-02-18 08:48:54.000000000 -0500
>>
>> +++ FlowTracker_Collector       2014-02-18 09:09:58.000000000 -0500
>>
>> @@ -303,7 +303,7 @@
>>
>>
>>
>>                         # Set up silk data sources
>>
>>
>>
>> -                       $data_root_dir = $silk_data_directory ."/".
>> $device_name;
>>
>> +                       $data_root_dir = $silk_data_directory;
>>
>>
>>
>>                         $silk_flow_type = "";
>>
>>
>>
>> --- FlowViewer_Main.cgi.orig    2014-02-18 08:52:30.000000000 -0500
>>
>> +++ FlowViewer_Main.cgi 2014-02-18 09:09:58.000000000 -0500
>>
>> @@ -431,7 +431,7 @@
>>
>>                  $silk_flow_type =~ s/\s+//g;
>>
>>          }
>>
>>
>>
>> -        $data_root_dir = $silk_data_directory ."/". $device_name;
>>
>> +        $data_root_dir = $silk_data_directory;
>>
>>
>>
>>          # Prepare rwfilter start and end time parameters
>>
>>
>>
>>
>> I also found that security/silktools uses UTC by default, but has a
>> configuration option to enable localtime (
>> https://tools.netsa.cert.org/silk/faq.html#timestamp-mismatch).
>>
>> Here is a patch to the Makefile containing a config option for localtime:
>>
>>
>> --- /usr/ports/silktools/Makefile.orig  2014-02-18 09:29:28.000000000
>> -0500
>>
>> +++ /usr/ports/silktools/Makefile       2014-02-18 09:41:48.000000000
>> -0500
>>
>> @@ -23,6 +23,11 @@
>>
>>  USES=          perl5
>>
>>  USE_PERL5=     build
>>
>>
>> +HAS_CONFIGURE= yes
>>
>> +OPTIONS_DEFINE= LOCALTIME
>>
>> +LOCALTIME_DESC= Use localtime instead of UTC
>>
>> +
>>
>> +
>>
>>  MAN1=  mapsid.1 num2dot.1 rwaddrcount.1 rwappend.1 \
>>
>>         rwbag.1 rwbagbuild.1 rwbagcat.1 rwbagtool.1 \
>>
>>         rwcat.1 rwcount.1 rwcut.1 rwdedupe.1 rwfglob.1 \
>>
>> @@ -51,6 +56,13 @@
>>
>>                 rwsender.8
>>
>>
>>  NO_STAGE=      yes
>>
>> +
>>
>> +.include <bsd.port.options.mk>
>>
>> +
>>
>> +.if ${PORT_OPTIONS:MLOCALTIME}
>>
>> +CONFIGURE_ARGS+=--enable-localtime
>>
>> +.endif
>>
>> +
>>
>>  post-patch:
>>
>>         @${REINPLACE_CMD} -e 's|echo aout|echo elf|' ${WRKSRC}/configure
>>
>>
>>
>> Thanks,
>>
>>
>> Chad
>>
>
>
>
> Here is another patch for net-mgmt/flowview so sensor filtering works. I
> am not sure why, but this file is originally trying to use the exporter as
> the sensor for SiLK devices. This is interesting since the PDF above
> indicated that the @exporter array was only used for flow-tools, not SiLK
> but alas here it is using it. If anything I think it would make more sense
> to use the "device" as the sensor, especially since @ipfix_devices is
> already defined as a sensor per the documentation. To make matters worse it
> is grepping for the probes and not the sensors in order to populate the
> --sensors= flag.
>
>
>
> --- FlowViewer_Utilities.pm.orig 2014-02-18 12:52:42.000000000 -0500
>
> +++ FlowViewer_Utilities.pm 2014-02-18 13:50:09.000000000 -0500
>
> @@ -2339,50 +2339,50 @@
>
>
>
>   # Set up exporter address filtering, if any
>
>
>
> - if ($exporter ne "") {
>
> + if ($device_name ne "") {
>
>
>
> -        $exporter =~ s/\s+//g;
>
> - $num_include_probe = 0;
>
> - @valid_probes = ();
>
> +        $device_name =~ s/\s+//g;
>
> + $num_include_sensor = 0;
>
> + @valid_sensors = ();
>
>
>
> - # Get valid probes (exporters) from the sensor.conf file
>
> + # Get valid sensors (device_names) from the sensor.conf file
>
>
>
> - $probe_command = "cat $sensor_config_directory/sensor.conf | grep probe
> > $work_directory/valid_probes_$suffix";
>
> - system ($probe_command);
>
> + $sensor_command = "cat $sensor_config_directory/sensor.conf | grep
> sensor > $work_directory/valid_sensors_$suffix";
>
> + system ($sensor_command);
>
>
>
> - open (PROBES,"<$work_directory/valid_probes_$suffix");
>
> + open (PROBES,"<$work_directory/valid_sensors_$suffix");
>
>   while (<PROBES>) {
>
> - ($probe_label,$probe) = split(/\s+/,$_);
>
> - if ($probe_label eq "probe") { push (@valid_probes,$probe); }
>
> + ($sensor_label,$sensor) = split(/\s+/,$_);
>
> + if ($sensor_label eq "sensor") { push (@valid_sensors,$sensor); }
>
>   }
>
>
>
>   while ($still_more) {
>
>
>
> -                ($exporter_name) = split(/,/,$exporter);
>
> -                $start_char = length($exporter_name) + 1;
>
> -                $exporter = substr($exporter,$start_char);
>
> +                ($device_name_name) = split(/,/,$device_name);
>
> +                $start_char = length($device_name_name) + 1;
>
> +                $device_name = substr($device_name,$start_char);
>
>
>
> -                if (substr($exporter_name,0,1) eq "-") {
>
> -   &print_error("SiLK software does not support exclusion of Exporters
> (Sensors) at this time: -$exporter_name"); last;
>
> +                if (substr($device_name_name,0,1) eq "-") {
>
> +   &print_error("SiLK software does not support exclusion of Exporters
> (Sensors) at this time: -$device_name_name"); last;
>
>                  } else {
>
> -                 foreach $probe (@valid_probes) {
>
> - if ($exporter_name eq $probe) {
>
> - $num_include_probe++;
>
> - if ($num_include_probe < 2) {
>
> - $sensor_field .= $exporter_name;
>
> +                 foreach $sensor (@valid_sensors) {
>
> + if ($device_name_name eq $sensor) {
>
> + $num_include_sensor++;
>
> + if ($num_include_sensor < 2) {
>
> + $sensor_field .= $device_name_name;
>
>   } else {
>
> - $sensor_field .= "," . $exporter_name;
>
> + $sensor_field .= "," . $device_name_name;
>
>   }
>
>   }
>
>   }
>
>                   }
>
>
>
> -                if ($exporter eq "") { last; }
>
> +                if ($device_name eq "") { last; }
>
>          }
>
>
>
>   $sensor_field  = " --sensors=" . $sensor_field;
>
>
>
> -        $save_file .= "_" . $exporter_name;
>
> +        $save_file .= "_" . $device_name;
>
>   }
>
>
>
>   # Set up Next Hop IP filtering, if any
>
>

Not only are these previously patches I submitted needed, but the startup
scripts (e.g. tools/flowtracker_restart) that make the tracker useful are
not patched nor are they installed in /usr/local/etc/rc.d. So far I have
noticed incorrect paths and linux-isms in the su command.  I don't have
time to fix and patch these now, but thought I would pass this info along
since this port will not work for anyone looking to set this up.



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAHP1p-VPyvEShTJZcpfy_iS3SRWwdwhZ6wUfS15N5z1HBBKmKw>