Skip to content

feat(jdbc): add retry mechanism for Arrow and JSON result streams #4100

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
keshavdandeva wants to merge 7 commits into
base: main
Choose a base branch
from
Open

feat(jdbc): add retry mechanism for Arrow and JSON result streams #4100

keshavdandeva wants to merge 7 commits into from
+114 −48

Conversation

@keshavdandeva
Copy link
Contributor

@keshavdandeva keshavdandeva commented Feb 11, 2026

b/473629793

@product-auto-label product-auto-label bot added size: m Pull request size is medium. api: bigquery Issues related to the googleapis/java-bigquery API. labels Feb 11, 2026
@gemini-code-assist
Copy link
Contributor

gemini-code-assist bot commented Feb 11, 2026

Summary of Changes

Hello @keshavdandeva, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly enhances the robustness of the BigQuery JDBC driver by introducing comprehensive retry mechanisms for both Arrow and JSON result streams. These changes ensure that transient network issues or temporary service unavailability during data retrieval and pagination are automatically handled, leading to a more resilient and reliable data fetching experience for users.

Highlights

  • Arrow Stream Retry Mechanism: Implemented a retry mechanism for the Arrow result stream processing, handling RuntimeException and ApiException (specifically NOT_FOUND errors) with a configurable retry timeout and delay.
  • JSON Stream Pagination Retry Mechanism: Added retry logic to the JSON result stream pagination (runNextPageTaskAsync) to handle exceptions during listTableData calls, including timeout management and interruption handling.
  • Logging Improvements: Updated various LOG.log(Level.WARNING, ...) calls to use the more concise LOG.warning(...) method, improving log message formatting.
  • Unused Import Removal: Removed an unused import for java.util.logging.Level.
Changelog
  • google-cloud-bigquery-jdbc/src/main/java/com/google/cloud/bigquery/jdbc/BigQueryStatement.java
    • Added retry logic to the arrowStreamProcessor to re-attempt reading from the stream upon encountering runtime exceptions, with a configurable timeout and exponential backoff.
    • Introduced a rowsRead counter and offset in ReadRowsRequest to resume reading from the correct position after a retry in the Arrow stream.
    • Implemented retry functionality within the runNextPageTaskAsync method for JSON result set pagination, allowing listTableData calls to be retried on exceptions, excluding 404 errors.
    • Updated logging statements from LOG.log(Level.WARNING, ...) to LOG.warning(...) across several methods for improved clarity and consistency.
    • Removed the java.util.logging.Level import as it was no longer needed after logging statement updates.
Activity
  • No specific activity has been recorded for this pull request yet.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

@keshavdandeva
Copy link
Contributor Author

keshavdandeva commented Feb 11, 2026

/gemini review

gemini-code-assist[bot]
gemini-code-assist bot reviewed Feb 11, 2026
View reviewed changes
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a retry mechanism for both Arrow and JSON result streams, which is a great enhancement for resiliency. The implementation looks mostly correct, but I've found a few issues. There's a potential infinite loop in the retry logic for JSON streams under specific interruption conditions. Additionally, there are some inconsistencies in the new logging statements, and in several places, the change from LOG.log to LOG.warning has led to the loss of stack traces in the logs, which could make debugging more difficult. I've provided specific comments and suggestions to address these points.

gemini-code-assist[bot]
gemini-code-assist bot reviewed Feb 11, 2026
View reviewed changes
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a valuable retry mechanism for both Arrow and JSON result streams, which will improve the robustness of the JDBC driver against transient network issues. The implementation looks solid, with configurable timeout and delay settings. I've identified one high-severity issue in the JSON stream handling that could lead to a hanging thread, and I've also suggested some improvements to the logging statements for better readability and consistency. Please apply the logging feedback to all similar new log statements.

@keshavdandeva
Copy link
Contributor Author

keshavdandeva commented Feb 11, 2026

/gemini review

gemini-code-assist[bot]
gemini-code-assist bot reviewed Feb 11, 2026
View reviewed changes
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

The pull request introduces retry mechanisms for both Arrow and JSON result streams in BigQueryStatement.java. This enhances the robustness of data retrieval by handling transient network interruptions and session expirations. Additionally, logging for InterruptedException has been improved in several areas, providing more detailed information upon interruption.

@keshavdandeva
Copy link
Contributor Author

keshavdandeva commented Feb 11, 2026

/gemini review

gemini-code-assist[bot]
gemini-code-assist bot reviewed Feb 11, 2026
View reviewed changes
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a retry mechanism for both Arrow and JSON result streams, aiming to improve the robustness of data fetching. However, a critical vulnerability has been identified in the JSON stream handling within runNextPageTaskAsync. The current implementation lacks a finally block to ensure the end-of-stream marker is always sent, which can lead to permanent thread hangs and potential Denial of Service if exceptions occur during result fetching. Additionally, there are medium-severity issues in the Arrow stream implementation related to logging and exception handling consistency. Addressing these points is crucial for the reliability and security of the new retry mechanism.

@keshavdandeva keshavdandeva marked this pull request as ready for review February 11, 2026 12:46
@keshavdandeva keshavdandeva requested review from a team as code owners February 11, 2026 12:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@logachev logachev logachev left review comments

+1 more reviewer

@gemini-code-assist gemini-code-assist[bot] gemini-code-assist[bot] left review comments

Reviewers whose approvals may not affect merge requirements

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

api: bigquery Issues related to the googleapis/java-bigquery API. size: m Pull request size is medium.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@keshavdandeva @logachev