StorageExceptionConsistency?

[yujihoo]

Environment Details

• Eclipse Store Version: 1.2.0
• JDK version: 15
• OS: Ubuntu 20
• Used frameworks: Spring Framework (Java and Maven)

Describe the bug

If we have a channel count equals to 1, and the service is being accessed by more than one person to read and write into the channel_0, it shows the following error? Apart from increasing the channel count (how much should I increase as it might affect performance too?), is there any other way to solve this? I want to control the concurrency like Redis does, which makes a queue for all the incoming operations on the data instead of using multi-threads?

Error

Caused by: org.eclipse.store.storage.exceptions.StorageException: Problem in channel #0
	at org.eclipse.store.storage.types.StorageChannelTask$Abstract.checkForProblems(StorageChannelTask.java:106)
	at org.eclipse.store.storage.types.StorageChannelTask$Abstract.waitOnCompletion(StorageChannelTask.java:168)
	at org.eclipse.store.storage.types.StorageSystem$Default.startThreads(StorageSystem.java:333)
	at org.eclipse.store.storage.types.StorageSystem$Default.internalStartUp(StorageSystem.java:516)
	at org.eclipse.store.storage.types.StorageSystem$Default.start(StorageSystem.java:602)
	at org.eclipse.store.storage.types.StorageSystem$Default.start(StorageSystem.java:72)
	at org.eclipse.store.storage.embedded.types.EmbeddedStorageManager$Default.start(EmbeddedStorageManager.java:247)
	at org.eclipse.store.storage.embedded.types.EmbeddedStorageManager$Default.start(EmbeddedStorageManager.java:95)
	at org.eclipse.store.storage.embedded.types.EmbeddedStorage.createAndStartStorageManager(EmbeddedStorage.java:609)
	at org.eclipse.store.storage.embedded.types.EmbeddedStorage.start(EmbeddedStorage.java:495)
	at org.eclipse.store.storage.embedded.types.EmbeddedStorage.start(EmbeddedStorage.java:342)
	at com.nsl.dataservices.cache.eclipsestore.EclipseStoreService.getS3Manager(EclipseStoreService.java:62)
	at com.nsl.dataservices.cache.eclipsestore.EclipseStoreService.getStorageManager(EclipseStoreService.java:46)
	at com.nsl.dataservices.cache.eclipsestore.EclipseStoreService.<clinit>(EclipseStoreService.java:34)
	... 32 more
Caused by: org.eclipse.store.storage.exceptions.StorageExceptionIoWriting
	at org.eclipse.store.storage.types.StorageFileWriter.truncateFile(StorageFileWriter.java:168)
	at org.eclipse.store.storage.types.StorageFileWriter.truncate(StorageFileWriter.java:144)
	at org.eclipse.store.storage.types.StorageFileManager$Default.handleLastFile(StorageFileManager.java:1291)
	at org.eclipse.store.storage.types.StorageFileManager$Default.initializeForExistingFiles(StorageFileManager.java:998)
	at org.eclipse.store.storage.types.StorageFileManager$Default.initializeStorage(StorageFileManager.java:873)
	at org.eclipse.store.storage.types.StorageChannel$Default.initializeStorage(StorageChannel.java:782)
	at org.eclipse.store.storage.types.StorageChannelTaskInitialize$Default.succeed(StorageChannelTaskInitialize.java:200)
	at org.eclipse.store.storage.types.StorageChannelTaskInitialize$Default.succeed(StorageChannelTaskInitialize.java:34)
	at org.eclipse.store.storage.types.StorageChannelSynchronizingTask$AbstractCompletingTask.synchronizedComplete(StorageChannelSynchronizingTask.java:78)
	at org.eclipse.store.storage.types.StorageChannelSynchronizingTask$AbstractCompletingTask.complete(StorageChannelSynchronizingTask.java:126)
	at org.eclipse.store.storage.types.StorageChannelTask$Abstract.processBy(StorageChannelTask.java:260)
	at org.eclipse.store.storage.types.StorageChannel$Default.work(StorageChannel.java:453)
	at org.eclipse.store.storage.types.StorageChannel$Default.run(StorageChannel.java:536)
	at java.base/java.lang.Thread.run(Thread.java:832)
Caused by: org.eclipse.store.storage.exceptions.StorageExceptionConsistency: Reopened file has inconsistent size: Expected size 2830020 != actual size 3478214.
	at org.eclipse.store.storage.types.StorageLiveDataFile$Default.internalOpenWriting(StorageLiveDataFile.java:282)
	at org.eclipse.store.storage.types.StorageFile$Abstract.ensureWritable(StorageFile.java:390)
	at org.eclipse.store.storage.types.StorageFile$Abstract.truncate(StorageFile.java:366)
	at org.eclipse.store.storage.types.StorageFileWriter.truncateFile(StorageFileWriter.java:164)
	... 13 more

[hg-ms]

Hello,
At the first glance the exception may be caused by multiple storage instance accessing one storage target. This must not happen.
Regarding concurrency and channels:

• Channels are an internal mechanism to parallelize IO operations of ONE store operation to multiple threads. If you don’t store huge amounts of objects (several thousands) in single store calls you should be fine using one channel. The channel count does not address concurrency in user code.
• The storage target must not be accessed by more than one storage instance!
  This applies to multiple processes and threads.
• If multithreading, you need take care of concurrency in your code to ensure that no other tread modifies the data that’s currently accessed by a storer, reading is ok. Reentrant Read write locks usual do a great job.

[yujihoo]

@hg-ms Is there any way if I want to access storage from multiple storage manager instances?

[hg-ms]

No, it’s not possible to access a storage target from multiple storage manager instances simultaneous except the limited Read-Only Mode.

[yujihoo]

Will EmbeddedStorageManager storageManager = EmbeddedStorage
.Foundation()
.setLockFileSetupProvider(Storage.LockFileSetupProvider())
.start();
mentioned in https://docs.eclipsestore.io/manual/storage/configuration/lock-file.html work? Facing an issue with the above code too.

[hg-ms]

The Lock File will help to prevent a second storage manager accessing the storage target. If a manager has locked the storage all others will throw an StorageExceptionInitialization at startup.
It may happen that the lock file is not created automatically, in that case you need to create an empty named “used.lock” in the storage directory manually.