Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

perf: use jobs.getQueryResults to download result sets #363

Merged
merged 7 commits into from
Nov 4, 2020
Merged

perf: use jobs.getQueryResults to download result sets #363

merged 7 commits into from
Nov 4, 2020

Conversation

tswast
Copy link
Contributor

@tswast tswast commented Nov 3, 2020 •
edited
Loading

Replaces tabledata.list when RowIterator is used for query results.
This likely also fixes a few edge cases around BigQuery scripting jobs.

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:

  • Make sure to open an issue as a bug/issue before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
  • Ensure the tests and linter pass
  • Code coverage does not decrease (if any source code was changed)
  • Appropriate docs were updated (if necessary)

Based on #361
Towards #362 🦕

@tswast
refactor: break job into multiple modules
ce147ef 
Original paths are retained for backwards compatibility.
@tswast
perf: use jobs.getQueryResults to download result sets
1957a7d 
Replaces `tabledata.list` when `RowIterator` is used for query results.
This likely also fixes a few edge cases around BigQuery scripting jobs.
@tswast tswast requested review from a team and shollyman November 3, 2020 22:06
@google-cla google-cla bot added the cla: yes This human has signed the Contributor License Agreement. label Nov 3, 2020
@product-auto-label product-auto-label bot added the api: bigquery Issues related to the googleapis/python-bigquery API. label Nov 3, 2020
@tswast tswast added the status: blocked Resolving the issue is dependent on other work. label Nov 3, 2020
@tswast tswast mentioned this pull request Nov 3, 2020
Closed
4 tasks
@tswast tswast removed the status: blocked Resolving the issue is dependent on other work. label Nov 4, 2020
@tswast tswast added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Nov 4, 2020
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Nov 4, 2020
shollyman
shollyman approved these changes Nov 4, 2020
View reviewed changes
Copy link
Contributor

@shollyman shollyman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. If you end up reusing _list_rows_from_query_results for the done polling you may need to consider that explicit project list, but it seems you're being distinct between the job polling and iteration usages here.

google/cloud/bigquery/client.py Show resolved Hide resolved
@tswast tswast merged commit 0c3476d into googleapis:master Nov 4, 2020
@tswast tswast deleted the issue362-getQueryResults-RowIterator branch November 4, 2020 19:42
@tswast tswast mentioned this pull request Nov 5, 2020
Closed
7 tasks
@release-please release-please bot mentioned this pull request Jan 4, 2022
Closed
@tswast tswast mentioned this pull request Nov 16, 2023
Merged
6 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@shollyman shollyman shollyman approved these changes

Assignees
No one assigned
Labels
api: bigquery Issues related to the googleapis/python-bigquery API. cla: yes This human has signed the Contributor License Agreement.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@tswast @shollyman @yoshi-kokoro