I have a very large table of IOT sample that I’m trying to run a relativly simple query against. Running the query normally using the MySql CLI returns a result in ~0.07 seconds. If I first prepare the query either via PDO or by running a SQL PREPARE statement then the request takes over a minute.
I’ve enabled the the optimizer trace feature, and it looks like when the statement is prepared, MySql ignores the index that it should use and does a file sort of the whole table. I’d like any insight if I am doing something wrong or if this looks like a MySql bug.
The table itself contains over 100 million samples, and at least 300 thousand are associated with the device being queried here. I ran these tests with MySql 8.0.23, but when I upgraded to 8.0.25 the issues persisted.
Table definition (some data rows ommited)
Create Table: CREATE TABLE `samples` ( `id` bigint unsigned NOT NULL AUTO_INCREMENT, `organization_id` int unsigned NOT NULL, `device_id` int unsigned NOT NULL, `created_at` timestamp NULL DEFAULT NULL, `raw_reading` int DEFAULT NULL, PRIMARY KEY (`id`), KEY `samples_organization_id_foreign` (`organization_id`), KEY `samples_reverse_device_id_created_at_organization_id_index` (`device_id`,`created_at` DESC,`organization_id`), CONSTRAINT `samples_device_id_foreign` FOREIGN KEY (`device_id`) REFERENCES `devices` (`id`) ON DELETE RESTRICT ON UPDATE CASCADE, CONSTRAINT `samples_organization_id_foreign` FOREIGN KEY (`organization_id`) REFERENCES `organizations` (`id`) ON DELETE CASCADE ON UPDATE CASCADE ) ENGINE=InnoDB AUTO_INCREMENT=188315314 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci
Sql That runs in < 1s
select * from `samples` where `samples`.`device_id` = 5852 and `samples`.`device_id` is not null and `id` != 188315308 order by `created_at` desc limit 1;
Sql That runs in over a minute
prepare test_prep from 'select * from `samples` where `samples`.`device_id` = ? and `samples`.`device_id` is not null and `id` != ? order by `created_at` desc limit 1'; set @a = 5852; set @b = 188315308; execute test_prep using @a, @b;
Trace for the non prepared SQL can be found at my gist, but the relevant part is
{ "reconsidering_access_paths_for_index_ordering": { "clause": "ORDER BY", "steps": [ ], "index_order_summary": { "table": "`samples`", "index_provides_order": true, "order_direction": "asc", "index": "samples_reverse_device_id_created_at_organization_id_index", "plan_changed": false } } },
Trace for the prepared query can be found at my other gist, but the relevant part is
{ "reconsidering_access_paths_for_index_ordering": { "clause": "ORDER BY", "steps": [ ], "index_order_summary": { "table": "`samples`", "index_provides_order": false, "order_direction": "undefined", "index": "samples_reverse_device_id_created_at_organization_id_index", "plan_changed": false } } },
Advertisement
Answer
Not full solution, but a workaround. I added an index on just my timestamp and that seems to satisfy the optimizer.
KEY `samples_created_at_index` (`created_at` DESC),
I’m going to try to clean up a minimal test case and post it over on MySql bugs. I’ll add a followup here if anything comes of that.