Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

IllegalArgumentException while calling getStreamSegmentInfo. #7250

Open
Bhupender-Y opened this issue Sep 8, 2023 · 2 comments
Open

IllegalArgumentException while calling getStreamSegmentInfo. #7250

Bhupender-Y opened this issue Sep 8, 2023 · 2 comments
Labels

Comments

@Bhupender-Y
Copy link
Contributor

Describe the bug
IllegalArgumentException while calling getStreamSegmentInfo.
To Reproduce
Write the event into stream and parallelly call getStreamSegmentInfo. Run this code in loop, will get this situation

     while (true) {
           streamName = UUID.randomUUID().toString();
           streamManager.createScope("scope");
           streamManager.createStream("scope", streamName, null);
           EventWriterConfig eventWriterConfig = EventWriterConfig.builder()
                                                                  .transactionTimeoutTime(60000)
                                                                  .build();
           EventStreamWriter<String> eventStreamWriter = clientFactory.createEventWriter(streamName, new JavaSerializer<>(), eventWriterConfig);
           new Thread(() -> eventStreamWriter.writeEvent(event).join()).start();
           long length = 0;
           while (length <= 0) {
               length = store.getStreamSegmentInfo(NameUtils.getQualifiedStreamSegmentName("scope", streamName, 0), Duration.ofMinutes(1)).get().getLength();
           }
       }

Screenshots
N/A

Additional information
Add any other information about the problem here, including the root cause if known.

@tkaitchuck
Copy link
Member

tkaitchuck commented Sep 8, 2023

An example stack trace:

java.lang.IllegalArgumentException: length must be a non-negative number and greater than startOffset.
	at com.google.common.base.Preconditions.checkArgument(Preconditions.java:142)
	at io.pravega.segmentstore.contracts.StreamSegmentInformation.<init>(StreamSegmentInformation.java:78)
	at io.pravega.segmentstore.contracts.StreamSegmentInformation$StreamSegmentInformationBuilder.build(StreamSegmentInformation.java:74)
	at io.pravega.segmentstore.server.containers.StreamSegmentMetadata.getSnapshot(StreamSegmentMetadata.java:377)
	at io.pravega.segmentstore.server.containers.MetadataStore.getSegmentSnapshot(MetadataStore.java:315)
	at io.pravega.segmentstore.server.containers.MetadataStore.getSegmentInfo(MetadataStore.java:284)
	at io.pravega.segmentstore.server.containers.StreamSegmentContainer.getStreamSegmentInfo(StreamSegmentContainer.java:512)
	at io.pravega.segmentstore.server.store.StreamSegmentService.lambda$7(StreamSegmentService.java:119)
	at io.pravega.segmentstore.server.store.SegmentContainerCollection.invoke(SegmentContainerCollection.java:94)
	at io.pravega.segmentstore.server.store.SegmentContainerCollection.invoke(SegmentContainerCollection.java:70)
	at io.pravega.segmentstore.server.store.StreamSegmentService.getStreamSegmentInfo(StreamSegmentService.java:117)

This is also causing build failures:
https://github.com/pravega/pravega/actions/runs/6116044066/job/16600711436

@tkaitchuck
Copy link
Member

This is caused by a race condition due to a bug in segment store's design.
In SegmentMetadataUpdateTransaction there is a method apply which applies the relevant updates to the UpdateableSegmentMetadata in this case StreamSegmentMetadata (however the bug exists for all implementations).
In each of these metadata implementations have synchronized methods which update one field at a time, and the apply method updates one field at a time.
The result is that if a read occurs after one field has been updated but before another, the object is in an inconsistent state. Instead the UpdateableSegmentMetadata interface should define a method to update many fields at once with their replacement values. (Or perhaps instead of updating the same object in memory a new object could just be constructed, then the object would be immutable, and it would be up to the SegmentMetadataUpdateTransaction to just replace the object with a new one.

The same bug exists for ContainerMetadataUpdateTransaction

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants