Skip to content

wip: respect order_by and limit on initial query. #2793

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

thruflo
Copy link
Contributor

@thruflo thruflo commented Jun 2, 2025

The server-side of the on-site hackathon work on respecting order_by and limit on initial shape query. @balegas added equivalent client-side support.

Copy link

codecov bot commented Jun 2, 2025

❌ 9 Tests Failed:

Tests completed Failed Passed Skipped
3106 9 3097 0
View the top 3 failed test(s) by shortest run time
Elixir.Electric.Postgres.ReplicationClientTest::test ReplicationClient against real db (toast) detoasts column values in updates
Stack Traces | 0s run time
10) test ReplicationClient against real db (toast) detoasts column values in updates (Electric.Postgres.ReplicationClientTest)
     .../electric/postgres/replication_client_test.exs:322
     ** (EXIT from #PID<0.5872.0>) killed
Elixir.Electric.Shapes.ShapeTest::test JSON should serialize shape with complex columns with backwards compatibility
Stack Traces | 0.0001s run time
13) test JSON should serialize shape with complex columns with backwards compatibility (Electric.Shapes.ShapeTest)
     .../electric/shapes/shape_test.exs:466
     ** (FunctionClauseError) no function clause matching in Electric.Shapes.Shape.from_json_safe/1

     The following arguments were given to Electric.Shapes.Shape.from_json_safe/1:

         # 1
         %{"root_table" => ["public", "foo"], "root_table_id" => 1, "selected_columns" => nil, "table_info" => [[["public", "foo"], %{"columns" => [%{"name" => "first", "type" => "text"}, %{"name" => "second", "type" => "text"}, %{"name" => "third", "type" => "text"}, %{"name" => "fourth", "type" => "text"}], "pk" => ["first", "second", "third"]}]], "where" => nil}

     code: assert {:ok, shape_old_decoded} = Shape.from_json_safe(shape_old_json)
     stacktrace:
       (electric 1.0.17) .../electric/shapes/shape.ex:452: Electric.Shapes.Shape.from_json_safe/1
       .../electric/shapes/shape_test.exs:502: (test)
Elixir.Electric.Shapes.ShapeTest::test JSON should serialize shape with complex columns with backwards compatibility
Stack Traces | 0.0002s run time
20) test JSON should serialize shape with complex columns with backwards compatibility (Electric.Shapes.ShapeTest)
     .../electric/shapes/shape_test.exs:466
     ** (FunctionClauseError) no function clause matching in Electric.Shapes.Shape.from_json_safe/1

     The following arguments were given to Electric.Shapes.Shape.from_json_safe/1:

         # 1
         %{"root_table" => ["public", "foo"], "root_table_id" => 1, "selected_columns" => nil, "table_info" => [[["public", "foo"], %{"columns" => [%{"name" => "first", "type" => "text"}, %{"name" => "second", "type" => "text"}, %{"name" => "third", "type" => "text"}, %{"name" => "fourth", "type" => "text"}], "pk" => ["first", "second", "third"]}]], "where" => nil}

     code: assert {:ok, shape_old_decoded} = Shape.from_json_safe(shape_old_json)
     stacktrace:
       (electric 1.0.17) .../electric/shapes/shape.ex:452: Electric.Shapes.Shape.from_json_safe/1
       .../electric/shapes/shape_test.exs:502: (test)

To view more test analytics, go to the Test Analytics Dashboard
📋 Got 3 mins? Take this short survey to help us improve Test Analytics.

@balegas
Copy link
Contributor

balegas commented Jun 2, 2025

do we want to merge this now? I'll can add the ts side of it after fixing tests

@thruflo
Copy link
Contributor Author

thruflo commented Jun 2, 2025

No, we definitely don't want to merge this as is. I think the generic sub-query approach we designed on day 2 is a much more viable approach. I've pushed this just so the code exists somewhere as a reference if we want to see what was involved in threading the new query params through.

@msfstef
Copy link
Contributor

msfstef commented Jul 3, 2025

Would this also not imply that the order_by and limit are respected by subsequent log updates? It's simple to apply it to the initial snapshot but that doesn't feel like much of a guarantee, especially when we compact etc

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants