DBD::MariaDB support

[bit0mike]

Trying to use DBD::MariaDB in place of DBD::mysql results in some weird behavior sometimes, notably Perl crashing (SIGBUS core dump, on FreeBSD at least) when disconnecting from the database when mixed with Parallel::ForkManager. Obviously normally Parallel::ForkManager shouldn't be used with open DBI handles, but that's one of the things DBIx::Connector is supposed to make "safe". And with the DBD::mysql driver, it is.

I've not tried making my own DBIx::Connector driver for MariaDB by copying the MySQL one yet.

The main difference between the two drivers is their handling of Unicode strings.

[theory]

Would you like to submit a PR to add it?

[bit0mike]

Looks like all it takes is to copy DBIx::Connector::Driver::mysql to DBIx::Connector::Driver::MariaDB and search/replace all "mysql" and "MySQL" strings to "MariaDB", except "mysql_auto_reconnect" becomes "mariadb_auto_reconnect". I'll put together the whole file tomorrow. Just having this there fixes a big performance problem I was having (disconnects were 2x slower) though I don't know yet if it fixes the SIGBUS issue on disconnect.

[karenetheridge]

Why would there be a performance improvement from making this change?

[Perlover]

Would you like to submit a PR to add it?

Hello!

I did this: #52

Can you check up this and to approve and to publish?

Thanks!

[bit0mike]

Looks good to me. Sorry I sat on that for so long.

[bit0mike]

Back to the coredump issue, I finally have a small script to reproduce the issue, at the risk of making me look like an idiot for doing something dumb:

#!/usr/local/bin/perl
use v5.38;
use Parallel::ForkManager;
use File::Temp;
use DBIx::Connector;
my $user = 'REDACTED';
my $pass = 'REDACTED';
my $forkmgr = Parallel::ForkManager->new( 32, File::Temp->newdir() );
my $dbix_obj = DBIx::Connector->new(
"DBI:MariaDB:database=test;host=localhost;mariadb_connect_timeout=10", $user, $pass, # dumps core sometimes
# "DBI:mysql:database=test;host=localhost;mysql_connect_timeout=10", $user, $pass, # does not dump core ever
{ AutoCommit => 1, RaiseError => 1, ShowErrorStatement => 1, Callbacks => {} }
);
$dbix_obj->mode('fixup');
$dbix_obj->dbh; # Comment out to stop core dumps with DBD::MariaDB
foreach my $c ( 0 .. 5000 ) {
unless ( $forkmgr->start ) {
my ($q) = $dbix_obj->run( fixup => sub { $_->selectall_array( 'SELECT MD5(?)', undef, $c ) } );
say "pid=$$ c=$c result=$$q[0]";
$forkmgr->finish(0);
}
}
$forkmgr->wait_all_children;

On FreeBSD 14.1 this code will create MANY coredumps, all apparently when the child process DB handles try to disconnect. The code does run successfully anyway (but obviously slowly because of all the core files being written) because the crash is when the child is trying to exit anyway.

On Ubuntu 24.04 this code instead prints this at every DB disconnect -- but not every time the program runs, just sometimes:

panic: DBI active kids (-1) < 0 or > kids (0) at /usr/lib/x86_64-linux-gnu/perl5/5.38/DBI.pm line 759.

To stop that warning (Ubuntu) or coredumps (FreeBSD) any one of the following works:

• use DBD::mysql (version 4.x; 5.x doesn't work w/ MariaDB anymore) instead of DBD::MariaDB. I don't know why there'd be a difference between the two, but, there is. This is why I went down the path of adding DBIx::Connector::Driver::MariaDB in the first place, to see if it would help. It did not, but it's good to have anyway.

OR

• move DBIx::Connector->new inside the $forkmgr->start block, which I know is more "correct" to do (forking with open handles of ANY kind is bad, after all), but making it fork-safe outside the block is one of the whole points of using DBIx::Connector in the first place

OR

• remove the $dbix_obj->dbh call immediately after connecting. I was doing this to make sure it could really connect, vs trying to do it lazily later on -- if the DB is really truly down, I want to know up front instead of later on. There could be a better way of accomplishing this test though (ping?), if this is a seriously stupid thing for me to be doing this way, I'm open to suggestions

[ap]

@bit0mike

1. Can you take a look at branch mariadb? I don’t actually expect that that might not work, but I don’t have MariaDB installed to test it with and I’m fairly busy right now – so I’d be grateful if you can do the doublechecking honors for me.

2. Could the crash be a DBD::MariaDB bug that has to be fixed there? That would be my instinct – but your test case does involve DBIx::Connector so I’m not entirely sure. Maybe there’s a connection attribute that DBIx::Connector should be setting, or something to that effect? Skimming the DBD::MariaDB docs doesn’t turn up anything of the sort so maybe not, but that hardly rises to the level of exhaustive examination of the issue.

[bit0mike]

1. No difference

2. Quite possible, and I mistakenly thought I had opened a duplicate bug report over there. It's now at Perl crashes when used with DBIx::Connector perl5-dbi/DBD-MariaDB#210

[ap]

Thank you!

No difference

Not confident what you mean here. No difference to what, or in what?

What I’m after here is an answer to this question: disregarding the crash bug, does the patch from branch mariadb work equivalently to what you suggested previously?

Looks like all it takes is to copy DBIx::Connector::Driver::mysql to DBIx::Connector::Driver::MariaDB and search/replace all "mysql" and "MySQL" strings to "MariaDB", except "mysql_auto_reconnect" becomes "mariadb_auto_reconnect". I'll put together the whole file tomorrow. Just having this there fixes a big performance problem I was having (disconnects were 2x slower) though I don't know yet if it fixes the SIGBUS issue on disconnect.

Does “no difference” mean “yes, it works the same as that, including the crash bug”?

[bit0mike]

Ok, testing with the master versions of DBD::MariaDB and DBIx::Connector, I'm not seeing any difference in behavior -- still core dumps on FreeBSD, still the same error (and an occasional not-noticed-until-now coredump) on Ubuntu.

The coredump on FreeBSD looks like this:

 (lldb) target create "/usr/local/bin/perl" --core "/var/tmp/perl.17317.1007.core"
 Core file '/var/tmp/perl.17317.1007.core' (x86_64) was loaded.
 (lldb) bt all
 * thread #1, name = 'perl', stop reason = signal SIGBUS
   * frame #0: 0x0000000800b1ee68 MariaDB.so`mariadb_db_close_mysql + 24
     frame #1: 0x0000000800b1f17b MariaDB.so`mariadb_dr_discon_all + 139
     frame #2: 0x0000000800b18435 MariaDB.so`XS_DBD__MariaDB__dr_discon_all_ + 117
     frame #3: 0x0000000800af8463 DBI.so`XS_DBI_dispatch + 8083
     frame #4: 0x00000008004f9550 libperl.so.5.40`___lldb_unnamed_symbol3485 + 2560
     frame #5: 0x0000000800575fd3 libperl.so.5.40`Perl_runops_standard + 67
     frame #6: 0x000000080046a0ce libperl.so.5.40`Perl_call_sv + 958
     frame #7: 0x0000000800466113 libperl.so.5.40`Perl_call_list + 467
     frame #8: 0x00000008004644ff libperl.so.5.40`perl_destruct + 367
     frame #9: 0x0000000000201da7 perl`main + 183
     frame #10: 0x00000008006d4a6a libc.so.7`__libc_start1 + 298
     frame #11: 0x0000000000201c80 perl`_start + 48

If anyone wants a run on either OS with DBI->trace(4) or something similar, let me know.

[bit0mike]

This fix seems to solve things: https://github.com/perl5-dbi/DBD-MariaDB/compare/master...choroba:inactive-destroy?expand=1

[ap]

Ok, testing with the master versions of DBD::MariaDB and DBIx::Connector, I'm not seeing any difference in behavior

That’s not surprising, I didn’t make any relevant changes on master. I’m asking about DBIx::Connector branch mariadb and your mention that having an actual MariaDB driver significantly speeds up disconnects. Can you confirm for me that the driver from branch mariadb works just as well as your modified copy of mysql.pm? (I would be quite surprised if it doesn’t, but I’d still like a double-check.)

[Perlover]

Hello everyone,

I would like to mention that I also use the DBIx::Connector module, which connects to MariaDB version 11.4 through the DBD::MariaDB driver. I wanted to point out that I also utilize multithreading through the Parallel::ForkManager module, and I am not experiencing any core dumps (I have high-load average there!)

The key difference from what has been discussed so far is that I establish the database connection from the forked child processes directly, not from the parent process. Naturally, if you establish a connection in the parent process and then try to use those connections in the child processes, it can lead to unpredictable behavior, including core dumps. I have encountered such issues in the past, even with a standard MySQL database and the DBD::MySQL driver.

Therefore, I don't see the benefit of connecting to the database from the parent process. The issue you're facing seems to stem from improper handling of processes.

[ap]

Release 0.60 with support for MariaDB is on its way to the CPAN. The crash appears to be a DBD::MariaDB issue and is fixed by a patch provided there. So it looks like there is nothing left to do here. Thanks to everyone for the helpful research and observations.