"MySQL server has gone away" on queue run #597

Coysh · 2021-10-07T20:27:54Z

Description
When some of my forms are submitted, they are added to the queue succesfully. But the problem is they are then stuck in the queue as "Reserved".

Looking at the log file, the error is to do with the MySQL server. But there are no other indications or reports across the site that the MySQL server has crashed.

2021-10-07 21:21:36 [-][1203][-][info][formie] Submission triggered for newBusinessBriefingForm.
2021-10-07 21:21:36 [-][1203][-][info][application] $_GET = [
    'p' => 'newbusiness'
]

$_POST = [
    'CRAFT_CSRF_TOKEN' => '••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••'
    'action' => 'formie/submissions/submit'
    'handle' => 'newBusinessBriefingForm'
    'fields' => [
        'clientName' => 'Tim Coysh Test'
        'clientContactLead' => 'Test'
        'clientEmailAddress' => 'tim@XXX'
        'activityType' => 'Retainer'
        'dateOfBrief' => [
            'date' => '2021-10-07'
        ]
        'estimatedBudget' => ''
        'projectName' => ''
        'durationOfProject' => ''
        'backgroundInformation' => 'Test'
        'marketingSupport' => ''
        'brandPositioningStatement' => ''
        'businessCharityObjectives' => 'Test'
        'whatDoYouHopeToAchieveByEngagingWithUs' => ''
        'prMessaging' => ''
        'primary' => ''
        'secondary' => ''
        'targetMedia' => ''
        'keySpokespeople' => ''
        'brandAmbassadors' => ''
        'competition' => ''
        'keyConsiderationsOrSensitivities' => ''
    ]
]
2021-10-07 21:21:37 [-][-][-][error][formie] Unable to update job info debug: “SQLSTATE[HY000]: General error: 2006 MySQL server has gone away
The SQL being executed was: UPDATE `craft_queue` SET `job`='O:34:\"verbb\\formie\\jobs\\SendNotification\":8:{s:12:\"submissionId\";i:61255;s:10:\"submission\";N;s:14:\"notificationId\";s:2:\"25\";s:12:\"notification\";N;s:5:\"email\";N;s:11:\"description\";N;s:30:\"\0craft\\queue\\BaseJob\0_progress\";i:0;s:35:\"\0craft\\queue\\BaseJob\0_progressLabel\";N;}' WHERE `id`='124649'” /home/515761.cloudwaysapps.com/hfvcruqxmf/public_html/vendor/yiisoft/yii2/db/Schema.php:678
2021-10-07 21:16:37 [-][-][-][info][application] $_GET = []

$_POST = []

Steps to reproduce
1.
2.

Form settings

Please pick from the options below.

Multi-page form: No
Submission Method: Ajax
Client-side Validation: Yes
Custom Form Templates: Yes

Additional info

Plugin version: 1.4.15
Craft version: 3.7.12
Multi-site: No

The text was updated successfully, but these errors were encountered:

engram-design · 2021-10-07T22:02:56Z

Can you share some details on your server setup (software, provider, specs), and MySQL version? It does sound like MySQL has fallen over, even briefly.

Coysh · 2021-10-07T22:14:20Z

Yep! Server is provisioned through CloudWays on a DO droplet. It’s a 1GB RAM, 1vCPU server. Looks like we’re on MySQL 5.5.5 and PHP 7.4 Tim

…

On 7 Oct 2021, at 23:03, Josh Crawford ***@***.***> wrote: Can you share some details on your server setup (software, provider, specs), and MySQL version? It does sound like MySQL has fallen over, even briefly. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.

engram-design · 2021-10-07T22:28:49Z

How's the general traffic to the site, is it under load? Not using anything like Blitz for static caching? This is a low-level Craft error, so will need to investigate it further, but this would be the first report of this sort of error.

Coysh · 2021-10-08T08:23:18Z

@engram-design Really low traffic.. Average load is close to 0.2 on the server.
We're not using Blitz or any caching, but it's a pretty basic static website so could install it - but unsure if that would fix the problem.

Coysh · 2021-10-08T09:15:28Z

@engram-design Hey Josh, also looks like this problem is only happening with one specific form.. This is the queue info:

:

engram-design · 2021-10-14T05:55:10Z

Hmmm, how large is the form? Judging from the fields in your initial log output, it doesn't seem like much.

Really difficult to tell, without seeing your form setup I think. Something is clearly taking up resources, but unsure as to what, or how best to pinpoint that without getting my hands on the project. Will keep tinkering.

engram-design · 2021-11-14T02:37:54Z

Closing due to inactivity

Coysh · 2021-11-25T12:19:35Z

@engram-design - I'm still getting this issue. I left it, as I thought it might be a server issue. But it does seem to only apply to formie submissions.

This is another extract from logs of last night:

2021-11-25 02:04:43 [-][-][-][error][formie] Unable to update job info debug: “SQLSTATE[HY000]: General error: 2006 MySQL server has gone away
The SQL being executed was: UPDATE `craft_queue` SET `job`='O:34:\"verbb\\formie\\jobs\\SendNotification\":8:{s:12:\"submissionId\";i:73693;s:10:\"submission\";N;s:14:\"notificationId\";s:2:\"32\";s:12:\"notification\";N;s:5:\"email\";N;s:11:\"description\";N;s:30:\"\0craft\\queue\\BaseJob\0_progress\";i:0;s:35:\"\0craft\\queue\\BaseJob\0_progressLabel\";N;}' WHERE `id`='56943'” /home/515761.cloudwaysapps.com/hfvcruqxmf/public_html/vendor/yiisoft/yii2/db/Schema.php:678
2021-11-25 01:59:42 [-][-][-][info][application] $_GET = []

$_POST = []
2021-11-25 02:04:47 [-][-][-][error][formie] Unable to update job info debug: “SQLSTATE[HY000]: General error: 2006 MySQL server has gone away
The SQL being executed was: UPDATE `craft_queue` SET `job`='O:34:\"verbb\\formie\\jobs\\SendNotification\":8:{s:12:\"submissionId\";i:73822;s:10:\"submission\";N;s:14:\"notificationId\";s:2:\"32\";s:12:\"notification\";N;s:5:\"email\";N;s:11:\"description\";N;s:30:\"\0craft\\queue\\BaseJob\0_progress\";i:0;s:35:\"\0craft\\queue\\BaseJob\0_progressLabel\";N;}' WHERE `id`='92987'” /home/515761.cloudwaysapps.com/hfvcruqxmf/public_html/vendor/yiisoft/yii2/db/Schema.php:678
2021-11-25 01:59:47 [-][-][-][info][application] $_GET = []

$_POST = []
2021-11-25 02:10:15 [-][-][-][error][formie] Unable to update job info debug: “SQLSTATE[HY000]: General error: 2006 MySQL server has gone away
The SQL being executed was: UPDATE `craft_queue` SET `job`='O:34:\"verbb\\formie\\jobs\\SendNotification\":8:{s:12:\"submissionId\";i:73693;s:10:\"submission\";N;s:14:\"notificationId\";s:2:\"32\";s:12:\"notification\";N;s:5:\"email\";N;s:11:\"description\";N;s:30:\"\0craft\\queue\\BaseJob\0_progress\";i:0;s:35:\"\0craft\\queue\\BaseJob\0_progressLabel\";N;}' WHERE `id`='56943'” /home/515761.cloudwaysapps.com/hfvcruqxmf/public_html/vendor/yiisoft/yii2/db/Schema.php:678
2021-11-25 02:05:15 [-][-][-][info][application] $_GET = []

$_POST = []
2021-11-25 02:15:15 [-][-][-][error][formie] Unable to update job info debug: “SQLSTATE[HY000]: General error: 2006 MySQL server has gone away
The SQL being executed was: UPDATE `craft_queue` SET `job`='O:34:\"verbb\\formie\\jobs\\SendNotification\":8:{s:12:\"submissionId\";i:73822;s:10:\"submission\";N;s:14:\"notificationId\";s:2:\"32\";s:12:\"notification\";N;s:5:\"email\";N;s:11:\"description\";N;s:30:\"\0craft\\queue\\BaseJob\0_progress\";i:0;s:35:\"\0craft\\queue\\BaseJob\0_progressLabel\";N;}' WHERE `id`='92987'” /home/515761.cloudwaysapps.com/hfvcruqxmf/public_html/vendor/yiisoft/yii2/db/Schema.php:678
2021-11-25 02:10:15 [-][-][-][info][application] $_GET = []

The site never has any downtime, and server load is always low.

engram-design · 2021-11-25T13:40:03Z

It must have something to do with the serialization of the submission when saving a sent notification I think, or potentially an infinite loop.

We might need to wait for #465 and #331 to get a copy of your form setup to see if I can replicate from that. Or, if you're able to sent through your craft_queue table to web at verbb.io I have a feeling it might be too large to handle.

Coysh · 2021-11-29T15:43:49Z

@engram-design - I only have one stuck item in the queue at the moment, but the same error is in the log.

:

engram-design · 2021-11-30T01:22:32Z

Yep, so Formie will try and create a serialization of the payload in the Job Data, but looks like it's not getting that far.

I would really need a copy of:

formie_forms with handle of yourFormHandle (whatever your form handle is)
formie_notifications with ID of 32
fmc_yourFormHandle with the ID of 74393

With that, I could hopefully replicate it, but I might still need your fields, which is tricky. If you can hold on, we might try the export feature in Formie 1.5.0 due in the next few weeks, which should be able to provide me everything I need to troubleshoot this one.

Coysh · 2021-11-30T11:24:07Z

Hey @engram-design - hopefully these work, but let me know if it's easier for me to send you over a DB dump of the whole site?

INSERT INTO `craft_formie_forms` (`id`, `handle`, `fieldContentTable`, `settings`, `templateId`, `submitActionEntryId`, `requireUser`, `availability`, `availabilityFrom`, `availabilityTo`, `availabilitySubmissions`, `defaultStatusId`, `dataRetention`, `dataRetentionValue`, `userDeletedAction`, `fileUploadsAction`, `fieldLayoutId`, `dateCreated`, `dateUpdated`, `uid`)
VALUES
	(61517, 'newBusinessBriefingForm2', '{{%fmc_newbusinessbriefingform2}}', '{\"displayFormTitle\":\"\",\"displayCurrentPageTitle\":\"\",\"displayPageTabs\":\"\",\"displayPageProgress\":\"\",\"progressPosition\":\"end\",\"defaultLabelPosition\":\"verbb\\\\formie\\\\positions\\\\AboveInput\",\"defaultInstructionsPosition\":\"verbb\\\\formie\\\\positions\\\\AboveInput\",\"submitMethod\":\"page-reload\",\"submitAction\":\"message\",\"submitActionTab\":null,\"submitActionUrl\":\"\",\"submitActionFormHide\":\"\",\"submitActionMessage\":\"[{\\\"type\\\":\\\"paragraph\\\",\\\"content\\\":[{\\\"type\\\":\\\"text\\\",\\\"text\\\":\\\"Submission saved.\\\"}]}]\",\"submitActionMessageTimeout\":\"\",\"submitActionMessagePosition\":\"top-form\",\"loadingIndicator\":\"\",\"loadingIndicatorText\":\"\",\"validationOnSubmit\":\"1\",\"validationOnFocus\":\"\",\"errorMessage\":\"[{\\\"type\\\":\\\"paragraph\\\",\\\"content\\\":[{\\\"type\\\":\\\"text\\\",\\\"text\\\":\\\"Couldn’t save submission due to errors.\\\"}]}]\",\"errorMessagePosition\":\"top-form\",\"availabilityMessage\":null,\"availabilityMessageDate\":null,\"availabilityMessageSubmissions\":null,\"integrations\":{\"mailchimp\":{\"enabled\":\"\",\"optInField\":\"\",\"listId\":\"\",\"fieldMapping\":\"\",\"appendTags\":\"\",\"useDoubleOptIn\":\"\"},\"recaptcha\":{\"enabled\":\"\",\"showAllPages\":\"\"},\"honeypot\":{\"enabled\":\"\",\"showAllPages\":\"\"}},\"submissionTitleFormat\":\"{timestamp}\",\"collectIp\":\"\",\"collectUser\":\"\",\"dataRetention\":null,\"dataRetentionValue\":null,\"fileUploadsAction\":null,\"redirectUrl\":null,\"defaultEmailTemplateId\":\"\",\"storeData\":null,\"userDeletedAction\":null}', 1, NULL, 0, 'always', NULL, NULL, NULL, 1, 'forever', NULL, 'retain', 'retain', 71, '2021-10-12 16:23:20', '2021-11-24 12:26:10', 'f6c0a722-eb88-46f2-b075-561c3ec46a75');

INSERT INTO `craft_formie_notifications` (`id`, `formId`, `templateId`, `name`, `enabled`, `subject`, `to`, `cc`, `bcc`, `replyTo`, `replyToName`, `from`, `fromName`, `content`, `attachFiles`, `enableConditions`, `conditions`, `dateCreated`, `dateUpdated`, `uid`)
VALUES
	(32, 61517, NULL, 'Email', 1, 'New Business form', '[email protected], [email protected], [email protected]', NULL, NULL, NULL, NULL, NULL, NULL, '[{\"type\":\"paragraph\",\"content\":[{\"type\":\"text\",\"text\":\"A user submission has been made on \\\"New Business Briefing\\\" at \"},{\"type\":\"variableTag\",\"attrs\":{\"label\":\"Timestamp (yyyy-mm-dd hh:mm:ss)\",\"value\":\"{timestamp}\"}}]},{\"type\":\"paragraph\",\"content\":[{\"type\":\"text\",\"text\":\"Their submission details are:\"}]},{\"type\":\"paragraph\",\"content\":[{\"type\":\"variableTag\",\"attrs\":{\"label\":\"All Form Fields\",\"value\":\"{allFields}\"}}]}]', 1, 0, '{\"sendRule\":\"send\",\"conditionRule\":\"all\",\"conditions\":[]}', '2021-10-12 16:33:07', '2021-11-24 12:26:10', '71fbd8f2-2ccd-450f-b00d-fd89d3128b5b');

INSERT INTO `craft_fmc_newbusinessbriefingform2` (`id`, `elementId`, `siteId`, `dateCreated`, `dateUpdated`, `uid`, `field_clientName_ryoaautv`, `field_clientContactLead_iwvjqzpr`, `field_clientEmailAddress_scocwgqt`, `field_activityType_gvltgafl`, `field_dateOfBrief_nfngdonv`, `field_estimatedBudget_osoneclr`, `field_backgroundInformation_qqmozqah`, `field_marketingSupport_qeooyqru`, `field_brandPositioningStatement_ylmjxppb`, `field_businessCharityObjectives_mmtdziqq`, `field_whatDoYouHopeToAchieveByEngagingJerseyRoadPr_xhyqzdox`, `field_prMessaging_oguuzshq`, `field_primary_hlfbqnvb`, `field_secondary_beubdkir`, `field_targetMedia_kpnrdhhs`, `field_keySpokespeople_qinfhcfm`, `field_brandAmbassadors_jkmlmyie`, `field_competition_uktecova`, `field_keyConsiderationsOrSensitivities_nyxtwfwq`)
VALUES
	(4, 74393, 2, '2021-11-25 12:13:47', '2021-11-25 12:13:47', '12a1774a-4334-4818-b760-cbef2c3a0efd', 'Tim Coysh', 'Tim', '[email protected]', 'Retainer', '2021-11-25 00:00:00', '', 'Test', '', '', 'Test', '', '', '', '', '', '', '', '', '');

engram-design · 2021-11-30T11:34:45Z

Thanks! I'll give that a shot and we'll see how we go...

Coysh · 2021-12-07T16:07:28Z

@engram-design - Thanks! Have you got any updates on this?

engram-design · 2021-12-07T21:49:07Z

Sorry @Coysh I've been preoccupied with the 1.5.0 release, I'll do my best to look at this today.

engram-design · 2021-12-08T22:45:20Z

@Coysh Okay, so it might be easier if you're up for sending your database in its entirety to web at verbb.io if that's possible?

Coysh · 2022-02-03T12:36:44Z

Hi @engram-design - any update on this?
I've had the following error on these at the moment:

Coysh · 2022-02-03T12:52:44Z

Hey @engram-design - just as an update, this issue looks similar: yiisoft/yii2-queue#439
And I got this error running the queue manually:

$ ./craft queue/run
^CError: SQLSTATE[HY000]: General error: 2006 MySQL server has gone away
The SQL being executed was: SELECT GET_LOCK('6308666b2496a1dc9724e644b98a4d744178264e', 3)

engram-design · 2022-02-03T22:45:59Z

I just can't seem to replicate it which is frustrating. I think from memory I even got your database, spun it up, and couldn't replicate it?

I think my last email back to you was Are you able to let me know how you run your queue on your server? - is there any special with your setup? I imagine you run the queue via a daemon, or form the command line (as opposed to automatically which is the default)?

Coysh · 2022-02-11T14:59:01Z

Hey Josh,

So I'm using the AsyncQueue plugin to run the queue

Tim

engram-design · 2022-02-26T22:49:23Z

I've not used AsyncQueue extensively, but will check to see if there's anything that might be different.

engram-design · 2022-02-27T02:23:00Z

So after extensively testing with AsyncQueue, I'm no further along this issue. I've also tested all your forms.

So that really leaves this as an environmental issue. Does this happen locally? Is there any chance of getting control panel and file-level access to your production site?

engram-design · 2022-06-08T05:17:07Z

Going to close this as I really, really can't reproduce this, and no other reports have come in for this issue in the 6 months since this was opened.

If we want to get to the bottom of it, reach out via Discord @crawf and I'll need to get hands-on access with your environment, as this seems clearly environmental.

engram-design closed this as completed Nov 14, 2021

engram-design reopened this Nov 25, 2021

engram-design closed this as not planned Won't fix, can't repro, duplicate, stale Jun 8, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

"MySQL server has gone away" on queue run #597

"MySQL server has gone away" on queue run #597

Coysh commented Oct 7, 2021

engram-design commented Oct 7, 2021

Coysh commented Oct 7, 2021 via email

engram-design commented Oct 7, 2021

Coysh commented Oct 8, 2021

Coysh commented Oct 8, 2021 •

edited

Loading

engram-design commented Oct 14, 2021

engram-design commented Nov 14, 2021

Coysh commented Nov 25, 2021 •

edited

Loading

engram-design commented Nov 25, 2021

Coysh commented Nov 29, 2021

engram-design commented Nov 30, 2021

Coysh commented Nov 30, 2021 •

edited

Loading

engram-design commented Nov 30, 2021

Coysh commented Dec 7, 2021 •

edited

Loading

engram-design commented Dec 7, 2021

engram-design commented Dec 8, 2021

Coysh commented Feb 3, 2022

Coysh commented Feb 3, 2022

engram-design commented Feb 3, 2022

Coysh commented Feb 11, 2022

engram-design commented Feb 26, 2022

engram-design commented Feb 27, 2022

engram-design commented Jun 8, 2022

"MySQL server has gone away" on queue run #597

"MySQL server has gone away" on queue run #597

Comments

Coysh commented Oct 7, 2021

Form settings

engram-design commented Oct 7, 2021

Coysh commented Oct 7, 2021 via email

engram-design commented Oct 7, 2021

Coysh commented Oct 8, 2021

Coysh commented Oct 8, 2021 • edited Loading

engram-design commented Oct 14, 2021

engram-design commented Nov 14, 2021

Coysh commented Nov 25, 2021 • edited Loading

engram-design commented Nov 25, 2021

Coysh commented Nov 29, 2021

engram-design commented Nov 30, 2021

Coysh commented Nov 30, 2021 • edited Loading

engram-design commented Nov 30, 2021

Coysh commented Dec 7, 2021 • edited Loading

engram-design commented Dec 7, 2021

engram-design commented Dec 8, 2021

Coysh commented Feb 3, 2022

Coysh commented Feb 3, 2022

engram-design commented Feb 3, 2022

Coysh commented Feb 11, 2022

engram-design commented Feb 26, 2022

engram-design commented Feb 27, 2022

engram-design commented Jun 8, 2022

Coysh commented Oct 8, 2021 •

edited

Loading

Coysh commented Nov 25, 2021 •

edited

Loading

Coysh commented Nov 30, 2021 •

edited

Loading

Coysh commented Dec 7, 2021 •

edited

Loading