fix: move grpc, proto-plus and protobuf packages to extras #1721

kiraksi · 2023-11-13T22:28:09Z

This PR:

Makes the package lighter weight by moving dependencies grpc, proto-plus and protobuff, which are needed for bigquery_v2, into optional extras
This will throw an import error if a user tries to access parts of bigquery_v2 without installing these packages, which they can then fix by importing them, as this version is deprecated

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:

Make sure to open an issue as a bug/issue before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
Ensure the tests and linter pass
Code coverage does not decrease (if any source code was changed)
Appropriate docs were updated (if necessary)

Fixes #1431 🦕

chalmerlowe

Thanks for the hard work. Appreciate the time and effort.
I added several comments.

setup.py

chalmerlowe · 2023-11-14T16:23:10Z

setup.py

@@ -29,20 +29,16 @@
 # 'Development Status :: 5 - Production/Stable'
 release_status = "Development Status :: 5 - Production/Stable"
 dependencies = [
-    "grpcio >= 1.47.0, < 2.0dev",  # https://github.com/googleapis/python-bigquery/issues/1262
-    "grpcio >= 1.49.1, < 2.0dev; python_version>='3.11'",
    # NOTE: Maintainers, please do not require google-api-core>=2.x.x


What happened to the grpcio install? Why have these been removed entirely?

I think we mentioned this at our last meeting, but it was because grpcio was both in dependencies and bqstorage optional extras, I believe it was supposed to be moved there but was not until the referenced issue above it was closed

tswast

LGTM!

We might actually want to add a system_noextras or move out the samples from samples/snippets that need extras (#790 (comment)) so that we have a little more confidence that these dependencies are actually optional.

I'm pretty sure they are, but it'd be good to verify with a bit more testing.

parthea

Consider changing the conventional commit message to fix: instead of chore: so that it's visible in the public release notes

) * fix: adds new property and tests * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * updates docs to correct a sphinx failure * Updates formatting * Update tests/system/test_query.py * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * Update google/cloud/bigquery/job/base.py * updates one test and uses int_or_none * Update tests/system/test_query.py testing something. * Update tests/system/test_query.py * testing coverage feature * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * minor edits * tweaks to noxfile for testing purposes * add new test to base as experiment * adds a test, updates import statements * add another test * edit to tests * formatting fixes * update noxfile to correct debug code * removes unneeded comments. --------- Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>

chalmerlowe · 2023-11-21T19:52:39Z

LGTM!

We might actually want to add a system_noextras or move out the samples from samples/snippets that need extras (#790 (comment)) so that we have a little more confidence that these dependencies are actually optional.

I'm pretty sure they are, but it'd be good to verify with a bit more testing.

@tswast was the intent to have this done as part of this PR OR create/piggyback off another Issue, such as 790?

tswast · 2023-11-21T20:12:18Z

@chalmerlowe I have synced with @kiraksi about this. I would like #1732 merged first, which makes sure we have some sample tests in samples/snippets with no other dependencies installed.

tswast

LGTM, assuming the samples/snippets tests succeed.

kiraksi · 2023-11-22T21:47:08Z

LGTM, assuming the samples/snippets tests succeed.

They succeeded when I tested before merge

…s#1721) * chore: move grpc, proto-plus and protobuff packages to extras * formatted with black * feat: add `job_timeout_ms` to job configuration classes (googleapis#1675) * fix: adds new property and tests * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * updates docs to correct a sphinx failure * Updates formatting * Update tests/system/test_query.py * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * Update google/cloud/bigquery/job/base.py * updates one test and uses int_or_none * Update tests/system/test_query.py testing something. * Update tests/system/test_query.py * testing coverage feature * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * minor edits * tweaks to noxfile for testing purposes * add new test to base as experiment * adds a test, updates import statements * add another test * edit to tests * formatting fixes * update noxfile to correct debug code * removes unneeded comments. --------- Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> --------- Co-authored-by: Chalmer Lowe <[email protected]> Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> Co-authored-by: Tim Swast <[email protected]>

…nto pandas extra (#1726) * feat: Introduce compatibility with native namespace packages * Update copyright year * removed pkg_resources from all test files and moved importlib into pandas extra * feat: removed pkg_resources from all test files and moved importlib into pandas extra * Adding no cover tag to test code * reformatted with black * undo revert * perf: use the first page a results when `query(api_method="QUERY")` (#1723) * perf: use the first page a results when `query(api_method="QUERY")` * add tests * respect max_results with cached page * respect page_size, also avoid bqstorage if almost fully downloaded * skip true test if bqstorage not installed * coverage * fix: ensure query job retry has longer deadline than API request deadline (#1734) In cases where we can't disambiguate API failure from job failure, this ensures we can still retry the job at least once. * fix: `load_table_from_dataframe` now assumes there may be local null values (#1735) Even if the remote schema is REQUIRED Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly: - [ ] Make sure to open an issue as a [bug/issue](https://togithub.com/googleapis/python-bigquery/issues/new/choose) before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea - [ ] Ensure the tests and linter pass - [ ] Code coverage does not decrease (if any source code was changed) - [ ] Appropriate docs were updated (if necessary) Fixes #1692 🦕 * chore: standardize samples directory - delete unneeded dependencies (#1732) * chore: standardize samples directory = delete unneeded dependencies * Removed unused import for linter * fix: move grpc, proto-plus and protobuf packages to extras (#1721) * chore: move grpc, proto-plus and protobuff packages to extras * formatted with black * feat: add `job_timeout_ms` to job configuration classes (#1675) * fix: adds new property and tests * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * updates docs to correct a sphinx failure * Updates formatting * Update tests/system/test_query.py * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * Update google/cloud/bigquery/job/base.py * updates one test and uses int_or_none * Update tests/system/test_query.py testing something. * Update tests/system/test_query.py * testing coverage feature * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * minor edits * tweaks to noxfile for testing purposes * add new test to base as experiment * adds a test, updates import statements * add another test * edit to tests * formatting fixes * update noxfile to correct debug code * removes unneeded comments. --------- Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> --------- Co-authored-by: Chalmer Lowe <[email protected]> Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> Co-authored-by: Tim Swast <[email protected]> * remove unnecessary version checks * undo bad commit, remove unneeded version checks * Revert "undo bad commit, remove unneeded version checks" This reverts commit 5c82dcf. * Revert "remove unnecessary version checks" This reverts commit 9331a7e. * revert bad changes, remove pkg_resources from file * after clarification, reimplement changes and ignore 3.12 tests * reformatted with black * removed minimum check * updated pandas installed version check --------- Co-authored-by: Anthonios Partheniou <[email protected]> Co-authored-by: Tim Swast <[email protected]> Co-authored-by: Chalmer Lowe <[email protected]> Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>

chore: move grpc, proto-plus and protobuff packages to extras

9381ef2

product-auto-label bot added size: s Pull request size is small. api: bigquery Issues related to the googleapis/python-bigquery API. labels Nov 13, 2023

formatted with black

25549ca

kiraksi self-assigned this Nov 13, 2023

kiraksi marked this pull request as ready for review November 13, 2023 23:11

kiraksi requested review from a team as code owners November 13, 2023 23:11

kiraksi requested review from prash-mi, tswast and chalmerlowe and removed request for prash-mi November 13, 2023 23:11

chalmerlowe reviewed Nov 14, 2023

View reviewed changes

tswast reviewed Nov 14, 2023

View reviewed changes

parthea changed the title ~~chore: move grpc, proto-plus and protobuff packages to extras~~ chore: move grpc, proto-plus and protobuf packages to extras Nov 15, 2023

parthea reviewed Nov 15, 2023

View reviewed changes

kiraksi changed the title ~~chore: move grpc, proto-plus and protobuf packages to extras~~ fix: move grpc, proto-plus and protobuf packages to extras Nov 16, 2023

product-auto-label bot added size: m Pull request size is medium. and removed size: s Pull request size is small. labels Nov 16, 2023

Merge branch 'main' into make-protobuf-optional

f5e107e

product-auto-label bot added size: s Pull request size is small. and removed size: m Pull request size is medium. labels Nov 18, 2023

kiraksi and others added 2 commits November 22, 2023 10:40

Merge branch 'main' into make-protobuf-optional

6397c73

Merge branch 'main' into make-protobuf-optional

fffcd8e

tswast mentioned this pull request Nov 22, 2023

provide "light" version of this package #1142

Closed

tswast approved these changes Nov 22, 2023

View reviewed changes

kiraksi merged commit 5ce4d13 into googleapis:main Nov 22, 2023
17 checks passed

kiraksi deleted the make-protobuf-optional branch November 22, 2023 21:39

release-please bot mentioned this pull request Nov 22, 2023

chore(main): release 3.14.0 #1709

Merged

This was referenced Dec 5, 2023

December 04, 2023 kitta65/bq-extension-vscode#257

Closed

December 04, 2023 kitta65/prettier-plugin-bq#264

Closed

December 04, 2023 kitta65/bq2cst#274

Closed

This was referenced Dec 19, 2023

December 18, 2023 kitta65/bq-extension-vscode#262

Closed

December 18, 2023 kitta65/prettier-plugin-bq#269

Closed

December 18, 2023 kitta65/bq2cst#279

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix: move grpc, proto-plus and protobuf packages to extras #1721

fix: move grpc, proto-plus and protobuf packages to extras #1721

kiraksi commented Nov 13, 2023 •

edited

Loading

chalmerlowe left a comment

chalmerlowe Nov 14, 2023

kiraksi Nov 14, 2023

tswast left a comment

parthea left a comment

chalmerlowe commented Nov 21, 2023

tswast commented Nov 21, 2023

tswast left a comment

kiraksi commented Nov 22, 2023

fix: move grpc, proto-plus and protobuf packages to extras #1721

fix: move grpc, proto-plus and protobuf packages to extras #1721

Conversation

kiraksi commented Nov 13, 2023 • edited Loading

chalmerlowe left a comment

Choose a reason for hiding this comment

chalmerlowe Nov 14, 2023

Choose a reason for hiding this comment

kiraksi Nov 14, 2023

Choose a reason for hiding this comment

tswast left a comment

Choose a reason for hiding this comment

parthea left a comment

Choose a reason for hiding this comment

chalmerlowe commented Nov 21, 2023

tswast commented Nov 21, 2023

tswast left a comment

Choose a reason for hiding this comment

kiraksi commented Nov 22, 2023

kiraksi commented Nov 13, 2023 •

edited

Loading