Skip to content
This repository was archived by the owner on Mar 13, 2026. It is now read-only.

fix: handle None when converting numerics to parquet#768

Merged
chalmerlowe merged 7 commits into
googleapis:mainfrom
Linchin:null-nums
May 29, 2024
Merged

fix: handle None when converting numerics to parquet#768
chalmerlowe merged 7 commits into
googleapis:mainfrom
Linchin:null-nums

Conversation

@Linchin

@Linchin Linchin commented May 13, 2024

Copy link
Copy Markdown
Contributor

decimal.Decimal reports an error when the input is None. This PR adds a lambda func to handle this case, as well as a unit test.

Fixes #719 🦕

@Linchin Linchin requested a review from chalmerlowe May 13, 2024 19:14
@Linchin Linchin requested review from a team May 13, 2024 19:14
@product-auto-label product-auto-label Bot added size: s Pull request size is small. api: bigquery Issues related to the googleapis/python-bigquery-pandas API. labels May 13, 2024
@Linchin Linchin added the owlbot:run Add this label to trigger the Owlbot post processor. label May 13, 2024
@gcf-owl-bot gcf-owl-bot Bot removed the owlbot:run Add this label to trigger the Owlbot post processor. label May 13, 2024
Comment thread pandas_gbq/load.py Outdated
# decimal.Decimal does not support `None` input, add support here.
# https://github.com/googleapis/python-bigquery-pandas/issues/719
def convert(x):
if x is None:

@chalmerlowe chalmerlowe May 15, 2024

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

QUESTION:

The original issue mentions that None and pd.NA are not handled appropriately.

This code seems to only address None.
Is the intent to check for both?

IF yes... this may be a possible check that is more general AND as other edge cases arise could enable edits to the code quickly and easily.

Suggested change
if x is None:
if x in {None, pd.NA}:

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed, thank you for catching this! I'll update the code.

Comment thread tests/unit/test_load.py
@Linchin Linchin added the owlbot:run Add this label to trigger the Owlbot post processor. label May 28, 2024
@gcf-owl-bot gcf-owl-bot Bot removed the owlbot:run Add this label to trigger the Owlbot post processor. label May 28, 2024

@chalmerlowe chalmerlowe left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Thanks for adding the extra test case.

@chalmerlowe chalmerlowe merged commit 53a4683 into googleapis:main May 29, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

api: bigquery Issues related to the googleapis/python-bigquery-pandas API. size: s Pull request size is small.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

nullable numeric column does not handled well

2 participants