Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid modifying input pixel coordinates in-place #16459

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

astrofrog
Copy link
Member

@astrofrog astrofrog commented May 15, 2024

As highlighted in #16244, modifying input pixel arrays in the WCSLIB C wrapper is not thread-safe. There are two possible solutions:

  • Option 1 Avoid modifying the input pixel arrays in-place, and instead copy the wcsprm, offset crpix, and then use that for the conversion. This has the added benefit that we also avoid modifying the wcsprm in-place to change NaN values to WCSLIB's UNDEFINED and back, which also has the risk of causing issues in a multi-threaded context (for instance if one thread changes UNDEFINED back to NaN while another thread is relying on the values being UNDEFINED).
  • Option 2 Always make sure a copy is made of the pixel coordinates in the WCS Python API before passing them to e.g. wcs.wcs.p2s. Currently if one calls wcs_pixel2world with (x, y, 0), a copy is made anyway to stack x and y. But in the case where it is called with (xy, 0), a copy is not made if the data is contiguous and of floating-point type. We could check for this case and force a copy.
  • Option 3 Force a copy when we parse the Numpy array into the C extension in this line:
  pixcrd = (PyArrayObject*)PyArray_ContiguousFromAny(pixcrd_obj, NPY_DOUBLE, 2, 2);

we could change this line to use PyArray_FromArray and pass a flag to force a copy, always.

Option 1 is this PR, although there are other places where this needs to be fixed, but I am waiting to see if we go with this option. It does mean copying the wcsprm but this should have a small memory footprint.

Option 2 is not really optimal because (a) users might call the astropy.wcs.Wcsprm methods directly in which case the unsafe behavior will continue, and (b) because it is nice to have a way to call the conversion routines which does

Option 3 should be safe and would require the fewest changes in lines of code, though it would increase memory usage, and would in particular result in unnecessary copies in some cases.

Performance-wise, Option 1 doesn't seem to be worse than the current code in main - we do require a copy of wcsprm but on the other hand we are not offsetting pixel coordinates. I'll be sure to add some benchmarks to astropy-benchmarks now though.

Anyway at this point, let's discuss if we like Option 1, and if so I can extend it to other parts of wcslib_wrap.c.

  • By checking this box, the PR author has requested that maintainers do NOT use the "Squash and Merge" button. Maintainers should respect this when possible; however, the final decision is at the discretion of the maintainer that merges the PR.

@github-actions github-actions bot added wcs external PRs and issues related to external packages vendored with Astropy (astropy.extern) labels May 15, 2024
Copy link

Thank you for your contribution to Astropy! 🌌 This checklist is meant to remind the package maintainers who will review this pull request of some common things to look for.

  • Do the proposed changes actually accomplish desired goals?
  • Do the proposed changes follow the Astropy coding guidelines?
  • Are tests added/updated as required? If so, do they follow the Astropy testing guidelines?
  • Are docs added/updated as required? If so, do they follow the Astropy documentation guidelines?
  • Is rebase and/or squash necessary? If so, please provide the author with appropriate instructions. Also see instructions for rebase and squash.
  • Did the CI pass? If no, are the failures related? If you need to run daily and weekly cron jobs as part of the PR, please apply the "Extra CI" label. Codestyle issues can be fixed by the bot.
  • Is a change log needed? If yes, did the change log check pass? If no, add the "no-changelog-entry-needed" label. If this is a manual backport, use the "skip-changelog-checks" label unless special changelog handling is necessary.
  • Is this a big PR that makes a "What's new?" entry worthwhile and if so, is (1) a "what's new" entry included in this PR and (2) the "whatsnew-needed" label applied?
  • At the time of adding the milestone, if the milestone set requires a backport to release branch(es), apply the appropriate "backport-X.Y.x" label(s) before merge.

Copy link

👋 Thank you for your draft pull request! Do you know that you can use [ci skip] or [skip ci] in your commit messages to skip running continuous integration tests until you are ready?

…d coordinates since this isn't thread-safe. Instead, make a copy of the wcsprm and offset crpix. Even when no offset is needed (e.g. if origin==1) it is a good idea to copy the wcsprm since we modify the NaN values in-place to change them to a WCSLIB-specific value.
@astrofrog astrofrog added this to the v7.0.0 milestone May 15, 2024
@astrofrog
Copy link
Member Author

Adding WCS benchmarks here: astropy/astropy-benchmarks#118

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
external PRs and issues related to external packages vendored with Astropy (astropy.extern) multi-threading needs-discussion wcs
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant