Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Speed of \JsonMachine\JsonMachine::fromFile #1

Closed
TechOverflow opened this issue Dec 5, 2018 · 5 comments
Closed

Speed of \JsonMachine\JsonMachine::fromFile #1

TechOverflow opened this issue Dec 5, 2018 · 5 comments

Comments

@TechOverflow
Copy link

Hi!

Is it normal for a response to take 8-9 seconds on a 32MB file? When using json_decode, it was maybe a second or two (at the cost of resources).

I am simply using
$array = \JsonMachine\JsonMachine::fromFile($file);
and matching an email with foreach to return all objects containing that email (usually 5-10 objects and 20kB or so).

Cheers

@halaxa
Copy link
Owner

halaxa commented Dec 5, 2018

Hello. Yes, it seems normal for most PCs.

If json_decode works better for you, use that. When it starts running out of memory, use JsonMachine.

The main advantage of this package is constant memory consumption and quite constant speed. Of course the speed can't mostly measure with native implementation, but that's not the goal. The goal is to be memory efficient on JSON streams of any length. In terms of speed, json_decode fluctuates greatly on huge files.

By the way, 5x slower than native (in your case) is not that bad. With JSON files with size of 100 MB and up it is sometimes even faster than json_decode. And while on 100 MB file json_decode eats about 1 GB of memory, JsonMachine is constantly at about 800 kB.

@halaxa halaxa changed the title Speed of \JsonMachine\JsonMachine::fromFile Speed of JsonMachine::fromFile Dec 5, 2018
@halaxa halaxa changed the title Speed of JsonMachine::fromFile Speed of \JsonMachine\JsonMachine::fromFile Dec 5, 2018
@TechOverflow
Copy link
Author

The delay is actually caused by the foreach, which is due to the 27 thousand objects. I'll try to cut down on the amount of data in each object to make it more efficient.

@halaxa
Copy link
Owner

halaxa commented Dec 5, 2018

Take a look at test/performance/testPerformace.php. You may get some hints. If you provided part or pseudo-part of your json file, we could discuss it here.

@halaxa
Copy link
Owner

halaxa commented Dec 17, 2018

Closing as not a bug for now.

@halaxa halaxa closed this as completed Dec 17, 2018
@LagShaggy LagShaggy mentioned this issue Mar 24, 2022
@halaxa
Copy link
Owner

halaxa commented May 6, 2023

This might be of help #97

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants