I have ran into this issue with a huge queue and I can't restart the container:
[12/06/27 19:36:47] ID=Broker02 (info) TCP_ACCEPTOR: accepting connections on tcp://b3isbqperfsonic02:2516
[12/06/27 19:36:47] ID=Broker02 (info) SonicMQ Broker started
com.odi.AbortException: An I/O exception occurred.
Stack trace at point of original failure:
*****************************************
java.io.IOException: No space left on device
at java.io.RandomAccessFile.writeBytes(Native Method)
at java.io.RandomAccessFile.write(Unknown Source)
at com.sonicsw.storage.impl.CachedPagesFile.writeDataPage(CachedPagesFile.java:289)
at com.sonicsw.storage.impl.CachedPagesFile.writePages(CachedPagesFile.java:282)
at com.sonicsw.storage.impl.PageManager.forceToDisk(PageManager.java:587)
at com.sonicsw.storage.impl.Storage.commit(Storage.java:285)
at com.odi.imp.sonic.StorageObject.commit(StorageObject.java:177)
at com.odi.imp.sonic.ObjectTable.commit(ObjectTable.java:95)
at com.odi.imp.sonic.Transaction.serverCommit(Transaction.java:61)
at com.odi.imp.Transaction.doPhasedCommit(Transaction.java:262)
at com.odi.imp.Transaction.doCommitOrCheckpoint(Transaction.java:203)
at com.odi.imp.Transaction.commitInternal(Transaction.java:166)
at com.odi.imp.Transaction.commit(Transaction.java:159)
at progress.message.db.pse.PSEDbContext.commit(PSEDbContext.java:295)
at progress.message.dbq.pse.DBQPSEBase.commit(DBQPSEBase.java:52)
at progress.message.broker.BrokerDatabase.commitQueueTran(BrokerDatabase.java:1252)
at progress.message.broker.QueueMsgSaver.threadMain(QueueMsgSaver.java:1198)
at progress.message.zclient.DebugThread.run(DebugThread.java:226)
*****************************************
at com.odi.imp.sonic.Server.IOFailure(Server.java:365)
at com.odi.imp.sonic.Transaction.serverCommit(Transaction.java:63)
at com.odi.imp.Transaction.doPhasedCommit(Transaction.java:262)
at com.odi.imp.Transaction.doCommitOrCheckpoint(Transaction.java:203)
at com.odi.imp.Transaction.commitInternal(Transaction.java:166)
at com.odi.imp.Transaction.commit(Transaction.java:159)
at progress.message.db.pse.PSEDbContext.commit(PSEDbContext.java:295)
at progress.message.dbq.pse.DBQPSEBase.commit(DBQPSEBase.java:52)
at progress.message.broker.BrokerDatabase.commitQueueTran(BrokerDatabase.java:1252)
at progress.message.broker.QueueMsgSaver.threadMain(QueueMsgSaver.java:1198)
at progress.message.zclient.DebugThread.run(DebugThread.java:226)
IO Exception flushing log file
PSEException: com.odi.AbortException: An I/O exception occurred.
Stack trace at point of original failure:
*****************************************
java.io.IOException: No space left on device
at java.io.RandomAccessFile.writeBytes(Native Method)
at java.io.RandomAccessFile.write(Unknown Source)
at com.sonicsw.storage.impl.CachedPagesFile.writeDataPage(CachedPagesFile.java:289)
at com.sonicsw.storage.impl.CachedPagesFile.writePages(CachedPagesFile.java:282)
at com.sonicsw.storage.impl.PageManager.forceToDisk(PageManager.java:587)
at com.sonicsw.storage.impl.Storage.commit(Storage.java:285)
at com.odi.imp.sonic.StorageObject.commit(StorageObject.java:177)
at com.odi.imp.sonic.ObjectTable.commit(ObjectTable.java:95)
at com.odi.imp.sonic.Transaction.serverCommit(Transaction.java:61)
at com.odi.imp.Transaction.doPhasedCommit(Transaction.java:262)
at com.odi.imp.Transaction.doCommitOrCheckpoint(Transaction.java:203)
at com.odi.imp.Transaction.commitInternal(Transaction.java:166)
at com.odi.imp.Transaction.commit(Transaction.java:159)
at progress.message.db.pse.PSEDbContext.commit(PSEDbContext.java:295)
at progress.message.dbq.pse.DBQPSEBase.commit(DBQPSEBase.java:52)
at progress.message.broker.BrokerDatabase.commitQueueTran(BrokerDatabase.java:1252)
at progress.message.broker.QueueMsgSaver.threadMain(QueueMsgSaver.java:1198)
at progress.message.zclient.DebugThread.run(DebugThread.java:226)
So I don't believe your message database or recovery logs are corrupt so I think you should be able to copy the files to a disk location with more space and restart the container/broker there. The recovery should then work.
If you are able to have the same pathing then it makes it easier. However if you change the pathing for the DB and recovery logs you will have to change the broker configuration before restarting.
We are currently trying to allocate more disk space. The (bogus) messages are just for testing. What files should we delete to have a fresh restart.
Thanks,
Aziz
You can delete the broker DB (default ./SonicMQStore) and recovery logs (./log) directories if you don't mind loosing any messages that are trapped there .. the broker will recreate the files when it restarts.