Skip to content

Preserve escape sequences when truncating SQL string literals - #898

Open
walitemuri wants to merge 1 commit into
andialbrecht:masterfrom
walitemuri:codex/preserve-escapes-in-truncated-strings
Open

Preserve escape sequences when truncating SQL string literals#898
walitemuri wants to merge 1 commit into
andialbrecht:masterfrom
walitemuri:codex/preserve-escapes-in-truncated-strings

Conversation

@walitemuri

Copy link
Copy Markdown

Summary

truncate_strings currently slices the raw contents of a string literal at an arbitrary character boundary. If that boundary bisects a doubled quote, the result is invalid SQL:

sqlparse.format("SELECT 'ab''cdef';", truncate_strings=3)
# Before: SELECT 'ab'[...]';
# After:  SELECT 'ab[...]';

There is also a related delimiter problem for literals beginning with an escaped quote: SELECT '''abcdef'; at width 3 becomes SELECT '''ab[...]'';. The initial doubled quote is content following the opening delimiter, not a two-character delimiter.

This change uses the actual single-quote delimiters and retains the longest prefix within the requested width that does not split a doubled quote or backslash escape. Strings that do not require truncation are unchanged. The truncation marker is still appended as supplied.

Validation

  • Added 36 parameterized regression cases covering boundaries before/after escapes, leading and consecutive escaped quotes, backslash escapes, newlines, unchanged short strings, and default/custom/empty markers. Five of the initial ten cases failed against the original code before the fix.
  • Full suite on Python 3.13: 542 passed, 2 xfailed, 1 xpassed. The same two xfails and one xpass are present with the original implementation (506 passed without the new cases).
  • Additional exhaustive check: executed 7,644 formatted SELECT statements with SQLite, using combinations of letters, quotes, and newlines with an empty marker; verified every result is a prefix of the original value and remains within the requested encoded width.
  • ruff check sqlparse/ and git diff --check pass.
  • Documented the safe-boundary behavior in the formatting API reference.

Checklist

  • ran the tests (pytest)
  • all style issues addressed (ruff)
  • changes are covered by tests
  • changes are documented

Prepared with OpenAI Codex assistance.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant