Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change: reduce number of SQL queries in GET_TASKS #2135

Draft
wants to merge 46 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
46 commits
Select commit Hold shift + click to select a range
88c2bff
Change: Check trash target references before accessing
mattmundell Jan 2, 2024
decd117
Move variables to where they are used
mattmundell Jan 8, 2024
4d55aa0
Combine CREDENTIAL_ITERATOR_COLUMNS and CREDENTIAL_ITERATOR_TRASH_COL…
mattmundell Jan 8, 2024
15661bf
Make credential_iterator_format_available variable use more obvious
mattmundell Jan 9, 2024
222f9aa
Note credential iterator field use
mattmundell Jan 9, 2024
fc8765d
Change: Use fewer queries in credential iterator when possible
mattmundell Jan 9, 2024
f0d4458
Declare variables where they are used
mattmundell Feb 7, 2024
2da21a6
Move get_task_schedule_xml into branches
mattmundell Feb 7, 2024
9fdaaa0
Move task_target into branch that uses it
mattmundell Feb 7, 2024
fb8882d
Move get_iterator_resource into branches
mattmundell Feb 7, 2024
e09e764
Move schedules_only case out to function
mattmundell Feb 7, 2024
65f515f
Use send instead of sendf for simple string
mattmundell Feb 7, 2024
2ff4aeb
Use new function sendf_to_client instead of obscure macro
mattmundell Feb 7, 2024
ef27916
Make sendf_to_client more convenient
mattmundell Feb 8, 2024
7487ae0
Make send_find_error_to_client more convenient
mattmundell Feb 8, 2024
b442180
Make send_to_client more convenient
mattmundell Feb 8, 2024
23590d7
Move sending of task out to function
mattmundell Feb 8, 2024
df91d62
Merge branch 'main' into faster-trash-tasks
mattmundell Feb 12, 2024
9fc89cf
Get get_tasks_send_task working
mattmundell Feb 12, 2024
f345791
Fix indent of large string
mattmundell Feb 12, 2024
9e8fbde
Make large string more manageable
mattmundell Feb 12, 2024
3bf8b72
Neaten start of get_tasks_send_task
mattmundell Feb 12, 2024
e97ac39
Move current_report setup out to function
mattmundell Feb 12, 2024
1f88070
Move last_report setup out to function
mattmundell Feb 12, 2024
230ed1c
Move sending of observers out to function
mattmundell Feb 12, 2024
bf71f23
Move sending of alerts out to function
mattmundell Feb 12, 2024
45edb72
Move sending of preferences out to function
mattmundell Feb 12, 2024
c2e368c
Define variables after declarations
mattmundell Feb 12, 2024
79e3085
Group more declarations
mattmundell Feb 12, 2024
b38ea71
Merge branch 'main' into faster-trash-tasks
mattmundell Feb 12, 2024
7b76bd9
Clang format
mattmundell Feb 12, 2024
d1e1689
Move task end tag back to get_tasks_send_task
mattmundell Feb 13, 2024
7e34dca
Use new function task_info to save queries in get_tasks_send_task
mattmundell Feb 13, 2024
483b923
Fix stray commas
mattmundell Feb 13, 2024
9977548
Move config_in_trash SQL into task_info
mattmundell Feb 19, 2024
88b02d4
Fix missing else
mattmundell Feb 19, 2024
9c02e89
Use WITH for config_in_trash
mattmundell Feb 19, 2024
c808fd9
Combine task_info SQL into single statement
mattmundell Feb 19, 2024
6f3f98a
Move second_last_report_id into task_info
mattmundell Feb 19, 2024
3e5716b
Move scanner_in_trash to task_info
mattmundell Feb 19, 2024
7b6e8c7
Move other scanner queries to task_info
mattmundell Feb 19, 2024
225b279
Fix: check if trash when getting scanner info in GET_TASKS
mattmundell Feb 19, 2024
14b1da6
Call scanner SQL directly in task_info
mattmundell Feb 20, 2024
a5f76b5
Move scanner SQL into main statement in task_info
mattmundell Feb 20, 2024
9f1322c
Actually set second_last_report_id in task_info
mattmundell Feb 20, 2024
70576f3
Merge branch 'main' into faster-trash-tasks-2
mattmundell Mar 7, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2,889 changes: 1,267 additions & 1,622 deletions src/gmp.c

Large diffs are not rendered by default.

55 changes: 46 additions & 9 deletions src/gmp_base.c
Original file line number Diff line number Diff line change
Expand Up @@ -125,12 +125,48 @@ buffer_xml_append_printf (GString *buffer, const char *format, ...)
* @return TRUE if send to client failed, else FALSE.
*/
gboolean
send_to_client (const char* msg,
int (*user_send_to_client) (const char*, void*),
void* user_send_to_client_data)
send_to_client (gmp_parser_t *parser, GError** error, const char* msg)
{
if (user_send_to_client && msg)
return user_send_to_client (msg, user_send_to_client_data);
if (parser
&& parser->client_writer
&& msg
&& parser->client_writer (msg, parser->client_writer_data))
{
error_send_to_client (error);
return TRUE;
}
return FALSE;
}

/**
* @brief Send a response message to the client.
*
* @param[in] gmp_parser GMP parser.
* @param[in] error Error.
* @param[in] format Format string.
*
* @return TRUE if error, else FALSE.
*/
gboolean
sendf_to_client (gmp_parser_t *parser, GError **error, const char *format, ...)
{
if (format)
{
va_list args;
gchar *msg;

va_start (args, format);
msg = g_markup_vprintf_escaped (format, args);
va_end (args);

if (parser->client_writer (msg, parser->client_writer_data))
{
g_free (msg);
error_send_to_client (error);
return TRUE;
}
g_free (msg);
}
return FALSE;
}

Expand All @@ -141,12 +177,14 @@ send_to_client (const char* msg,
* @param[in] type Resource type.
* @param[in] id Resource ID.
* @param[in] gmp_parser GMP Parser.
* @param[in] error Error.
*
* @return TRUE if out of space in to_client, else FALSE.
*/
gboolean
send_find_error_to_client (const char* command, const char* type,
const char* id, gmp_parser_t *gmp_parser)
send_find_error_to_client (const char *command, const char *type,
const char *id, gmp_parser_t *gmp_parser,
GError **error)
{
gchar *msg;
gboolean ret;
Expand All @@ -155,8 +193,7 @@ send_find_error_to_client (const char* command, const char* type,
STATUS_ERROR_MISSING
"\" status_text=\"Failed to find %s '%s'\"/>",
command, type, id);
ret = send_to_client (msg, gmp_parser->client_writer,
gmp_parser->client_writer_data);
ret = send_to_client (gmp_parser, error, msg);
g_free (msg);
return ret;
}
Expand Down
31 changes: 13 additions & 18 deletions src/gmp_base.h
Original file line number Diff line number Diff line change
Expand Up @@ -44,11 +44,14 @@ void
buffer_xml_append_printf (GString *, const char *, ...);

gboolean
send_to_client (const char *, int (*) (const char *, void *), void *);
send_to_client (gmp_parser_t *, GError **, const char *);

gboolean
sendf_to_client (gmp_parser_t *, GError **, const char *, ...);

gboolean
send_find_error_to_client (const char *, const char *, const char *,
gmp_parser_t *);
gmp_parser_t *, GError **);

void
error_send_to_client (GError **);
Expand All @@ -70,11 +73,9 @@ internal_error_send_to_client (GError **);
do \
{ \
gchar *msg = g_markup_printf_escaped (format, ##args); \
if (send_to_client (msg, gmp_parser->client_writer, \
gmp_parser->client_writer_data)) \
if (send_to_client (gmp_parser, error, msg)) \
{ \
g_free (msg); \
error_send_to_client (error); \
return err_ret; \
} \
g_free (msg); \
Expand All @@ -94,11 +95,9 @@ internal_error_send_to_client (GError **);
do \
{ \
gchar *msg = g_markup_printf_escaped (format, ##args); \
if (send_to_client (msg, gmp_parser->client_writer, \
gmp_parser->client_writer_data)) \
if (send_to_client (gmp_parser, error, msg)) \
{ \
g_free (msg); \
error_send_to_client (error); \
return; \
} \
g_free (msg); \
Expand All @@ -113,16 +112,12 @@ internal_error_send_to_client (GError **);
*
* @param[in] msg The message, a string.
*/
#define SEND_TO_CLIENT_OR_FAIL(msg) \
do \
{ \
if (send_to_client (msg, gmp_parser->client_writer, \
gmp_parser->client_writer_data)) \
{ \
error_send_to_client (error); \
return; \
} \
} \
#define SEND_TO_CLIENT_OR_FAIL(msg) \
do \
{ \
if (send_to_client (gmp_parser, error, msg)) \
return; \
} \
while (0)

void
Expand Down
15 changes: 5 additions & 10 deletions src/gmp_configs.c
Original file line number Diff line number Diff line change
Expand Up @@ -572,11 +572,8 @@ create_config_run (gmp_parser_t *gmp_parser, GError **error)
case 2:
if (send_find_error_to_client ("create_config", "config",
entity_text (copy),
gmp_parser))
{
error_send_to_client (error);
return;
}
gmp_parser, error))
return;
log_event_fail ("config", "Config", NULL, "created");
break;
case 99:
Expand Down Expand Up @@ -1042,11 +1039,9 @@ modify_config_run (gmp_parser_t *gmp_parser, GError **error)
if (send_find_error_to_client ("modify_config",
"config",
config_id,
gmp_parser))
{
error_send_to_client (error);
return;
}
gmp_parser,
error))
return;
log_event_fail ("config", "Scan Config", config_id, "modified");
return;
default:
Expand Down
7 changes: 2 additions & 5 deletions src/gmp_delete.c
Original file line number Diff line number Diff line change
Expand Up @@ -139,11 +139,8 @@ delete_run (gmp_parser_t *gmp_parser, GError **error)
break;
case 2:
if (send_find_error_to_client
(delete.command, delete.type, delete.id, gmp_parser))
{
error_send_to_client (error);
return;
}
(delete.command, delete.type, delete.id, gmp_parser, error))
return;
log_event_fail (delete.type, delete.type_capital,
delete.id,
"deleted");
Expand Down
43 changes: 19 additions & 24 deletions src/gmp_get.c
Original file line number Diff line number Diff line change
Expand Up @@ -261,15 +261,14 @@ get_next (iterator_t *resources, get_data_t *get, int *first, int *count,
/**
* @brief Send start of GET response.
*
* @param[in] gmp_parser GMP Parser.
* @param[in] error Error.
* @param[in] type Type.
* @param[in] write_to_client Function that sends to clients.
* @param[in] write_to_client_data Data for write_to_client.
*
* @return 0 success, 1 send to client failed.
*/
int
send_get_start (const char *type, int (*write_to_client) (const char*, void*),
void* write_to_client_data)
send_get_start (gmp_parser_t *gmp_parser, GError **error, const char *type)
{
gchar *msg;

Expand All @@ -285,7 +284,7 @@ send_get_start (const char *type, int (*write_to_client) (const char*, void*),
type);


if (send_to_client (msg, write_to_client, write_to_client_data))
if (send_to_client (gmp_parser, error, msg))
{
g_free (msg);
return 1;
Expand All @@ -309,8 +308,8 @@ send_get_start (const char *type, int (*write_to_client) (const char*, void*),
*/
int
send_get_common (const char *type, get_data_t *get, iterator_t *iterator,
int (*write_to_client) (const char *, void*),
void* write_to_client_data, int writable, int in_use)
gmp_parser_t *gmp_parser, GError **error, int writable,
int in_use)
{
GString *buffer;
const char *tag_type;
Expand Down Expand Up @@ -453,7 +452,7 @@ send_get_common (const char *type, get_data_t *get, iterator_t *iterator,
}
}

if (send_to_client (buffer->str, write_to_client, write_to_client_data))
if (send_to_client (gmp_parser, error, buffer->str))
{
g_string_free (buffer, TRUE);
return 1;
Expand Down Expand Up @@ -550,17 +549,16 @@ buffer_get_filter_xml (GString *msg, const char* type,
* @param[in] count Page count.
* @param[in] filtered Filtered count.
* @param[in] full Full count.
* @param[in] write_to_client Function that sends to clients.
* @param[in] write_to_client_data Data for write_to_client.
* @param[in] gmp_parser GMP Parser.
* @param[in] error Error.
*
* @return 0 success, 1 sending to client failed, 2 failed to allocate filter
* term.
*/
static int
send_get_end_internal (const char *type, get_data_t *get, int get_counts,
int count, int filtered, int full,
int (*write_to_client) (const char *, void*),
void* write_to_client_data)
gmp_parser_t *gmp_parser, GError **error)
{
gchar *sort_field, *filter;
int first, max, sort_order;
Expand Down Expand Up @@ -685,7 +683,7 @@ send_get_end_internal (const char *type, get_data_t *get, int get_counts,
g_free (sort_field);
g_free (filter);

if (send_to_client (msg->str, write_to_client, write_to_client_data))
if (send_to_client (gmp_parser, error, msg->str))
{
g_string_free (msg, TRUE);
return 1;
Expand All @@ -702,37 +700,34 @@ send_get_end_internal (const char *type, get_data_t *get, int get_counts,
* @param[in] count Page count.
* @param[in] filtered Filtered count.
* @param[in] full Full count.
* @param[in] write_to_client Function that sends to clients.
* @param[in] write_to_client_data Data for write_to_client.
* @param[in] gmp_parser GMP Parser.
* @param[in] error Error.
*
* @return 0 success, 1 sending to client failed, 2 failed to allocate filter
* term.
*/
int
send_get_end (const char *type, get_data_t *get, int count, int filtered,
int full, int (*write_to_client) (const char *, void*),
void* write_to_client_data)
int full, gmp_parser_t *gmp_parser, GError **error)
{
return send_get_end_internal (type, get, 1, count, filtered, full,
write_to_client, write_to_client_data);
gmp_parser, error);
}

/**
* @brief Send end of GET response, skipping result counts.
*
* @param[in] type Type.
* @param[in] get GET data.
* @param[in] write_to_client Function that sends to clients.
* @param[in] write_to_client_data Data for write_to_client.
* @param[in] gmp_parser GMP Parser.
* @param[in] error Error.
*
* @return 0 success, 1 sending to client failed, 2 failed to allocate filter
* term.
*/
int
send_get_end_no_counts (const char *type, get_data_t *get,
int (*write_to_client) (const char *, void*),
void* write_to_client_data)
gmp_parser_t *gmp_parser, GError **error)
{
return send_get_end_internal (type, get, 0, 0, 0, 0, write_to_client,
write_to_client_data);
return send_get_end_internal (type, get, 0, 0, 0, 0, gmp_parser, error);
}
Loading
Loading