-
Notifications
You must be signed in to change notification settings - Fork 6
/
README
168 lines (125 loc) · 4.88 KB
/
README
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
Logging to, and input from, PostgreSQL Databases
================================================
Introduction and Warning
========================
This plugin allows logging to as well as reading from PostgreSQL
databases. While the basic functionality seems to work, it has not
seen much real-life testing and no functionality guarantees are made.
This plugin should be considered experimental.
Installation
------------
After installing PostgreSQL, you can install the Zeek PostgreSQL module
either using zkg, or manually via the command-line.
To install the plugin using zkg, use
```console
# zkg install 0xxon/zeek-postgresql
```
To install manually from the cloned repository, use::
```console
# ./configure && make && make install
```
If PostgreSQL is installed in a non-standard location, add
``--with-postgresql=<postgresql-base-directory`` to the ``configure`` command.
Use zeek -N to verify correct installation:
```console
# zeek -N Johanna::PostgreSQL
Johanna::PostgreSQL - PostgreSQL log writer and input reader (dynamic, version 0.2.0)
```
Logging Data into PostgreSQL databases
-------------------------------------
The easiest way to add PostgreSQL logging is by adding a logging filter to an
already existing logging stream. This first example also sends the conn.log
to PostgreSQL:
```zeek
event zeek_init()
{
local filter: Log::Filter = [$name="postgres", $path="conn", $writer=Log::WRITER_POSTGRESQL, $config=table(["dbname"]="testdb")];
Log::add_filter(Conn::LOG, filter);
}
```
This will write to a database named testdb into the table named conn. Note that
the table will be automatically be created by the PostgreSQL plugin, if it does
not yet exist. If a table with the specified name already exists, it is used;
the existing columns have to be compatible with the column names and types that
the Zeek plugin expects.
Data can be read from PostgreSQL using a script similar to:
```zeek
redef exit_only_after_terminate = T;
type InfoType: record {
ts: time;
uid: string;
duration: interval;
};
event line(description: Input::EventDescription, tpe: Input::Event, r: InfoType)
{
print r;
}
event zeek_init()
{
Input::add_event([$source="select ts, uid, duration from conn;", $name="postgres", $fields=InfoType, $ev=line, $want_record=T,
$reader=Input::READER_POSTGRESQL, $config=table(["dbname"]="testdb")]);
}
event Input::end_of_data(name: string, source:string)
{
print "End of data";
terminate();
}
```
By default, the plugin connects to PostgreSQL as the user running Zeek,
without supplying any additional username or password.
Type mapping
============
The writer automatically maps the Zeek types to the following PostgreSQL data
types:
<table>
<tr>
<th>Zeek type</th>
<th>PostgreSQL type</th>
</tr><tr><td>Bool</td><td>boolean</td>
</tr><tr><td>int</td><td>bigint</td>
</tr><tr><td>count</td><td>bigint</td>
</tr><tr><td>port</td><td>bigint</td>
</tr><tr><td>addr</td><td>inet</td>
</tr><tr><td>subnet</td><td>inet</td>
</tr><tr><td>time</td><td>double precision</td>
</tr><tr><td>interval</td><td>double precision</td>
</tr><tr><td>double</td><td>double precision</td>
</tr><tr><td>enum</td><td>text</td>
</tr><tr><td>string</td><td>text/bytea</td>
</tr><tr><td>func</td><td>text/bytea</td>
</tr><tr><td>set[type]</td><td>type[]</td>
</tr><tr><td>vector[type]</td><td>type[]</td>
</tr>
</table>
For string and func, bytea is used if the $config option "bytea_instead_of_text"
is set.
Configuration options: PostgreSQL Writer
========================================
The PostgreSQL writer supports the following configuration options that can be
passed in $config:
- *hostname*: hostname to connect to
- *port*: port to connect to
- *dbname*: name of database to connect to
- *conninfo*: connection string using parameter key words as defined in
https://www.postgresql.org/docs/9.3/static/libpq-connect.html. Can be used
to pass usernames, passwords, etc. hostname, port, and dbname are ignored if
conninfo is specified.
Example: host=127.0.0.1 user=johanna
- *sql_addition*: SQL string that is appended to the insert statement
generated by the plugin. This can be used to specify a conflict clause
like: "ON CONFLICT DO NOTHING"
- *continue_on_errors*: ignore insert errors and do not kill the database
connection.
- *bytea_instead_of_text*: write strings/funcs to as bytea instead of text.
Configuration options: PostgreSQL Reader
========================================
The PostgreSQL reader supports the following configuration options that can be
passed in $config:
- *hostname*: hostname to connect to
- *port*: port to connect to
- *dbname*: name of database to connect to
- *conninfo*: connection string using parameter key words as defined in
https://www.postgresql.org/docs/9.3/static/libpq-connect.html. Can be used
to pass usernames, passwords, etc. hostname, port, and dbname are ignored if
conninfo is specified.
Example: host=127.0.0.1 user=johanna