Replaced unbounded thread pools in `ThreadManager` with bounded `ThreadPoolExecutor` to prevent memory exhaustion under load. Refactored shutdown logic to handle exceptions per-executor, set threads to daemon mode in `BaseThreadFactory`, and aligned synchronization patterns across multiple classes (`TaskManagerImpl`, `ChannelMessagePurger`, and `MusicManager`). Addressed lock cleanup races, double unlock issues, and static collection thread-safety concerns.
7.3 KiB
7.3 KiB
sessionId
| sessionId |
|---|
| session-260910-054543-zckm |
Requirements
Overview & Goals
Analyze and fix thread-safety issues and improve the ExecutorProvider / ThreadManager infrastructure and all code that interacts with it.
Scope
In Scope
- Fix thread-safety bugs in
ThreadManager,TaskManagerImpl,ChannelMessagePurger,MusicManager - Improve executor pool configuration (bounded pools, proper shutdown)
- Clean up inconsistent synchronization patterns
Out of Scope
- Rewriting the task/command architecture
- Changing the
QueueManagersynchronization (it's consistent and correct, just verbose)
Technical Design
Current Issues Found
1. ThreadManager (core/thread/ThreadManager.java)
- Unbounded cached thread pools —
newCachedThreadPool()creates unlimited threads; under load (e.g., many guilds purging simultaneously) this can exhaust memory. - Shutdown bug (line 66-69) — If
awaitTerminationthrowsInterruptedExceptionon the first executor, thecatchblock exits the loop and remaining executors only getshutdownNow()without a priorshutdown()call. Each executor should be shut down independently. - No daemon threads —
BaseThreadFactorydoesn't setthread.setDaemon(true), so leaked threads can prevent JVM exit.
2. TaskManagerImpl (core/tasks/TaskManagerImpl.java)
- Null values in
ConcurrentHashMap(line 178-184) —computeIfAbsentreturnsnullforPersistableTask/BuiltInTask, butConcurrentHashMapdoes not permit null values. This will throwNullPointerExceptionat runtime. - Mixed synchronization —
getTasks()issynchronizedonthis, but all mutations useConcurrentHashMapoperations (nosynchronized). Thesynchronizedkeyword ongetTasks()provides no actual protection sinceinsert(),removeTask(), etc. don't synchronize onthis. hasTask()race (line 93-95) — Check-then-act across two collections (tasksandpendingTasks) without synchronization.handle()lock cleanup race (line 191-194) — After acquiring the lock, it checkstasks.containsKey()and removes fromtasksLock, but another thread could be waiting on the same lock and will find it removed.
3. ChannelMessagePurger (utils/ChannelMessagePurger.java)
- Double unlock bug —
purgeMessages()acquireslockviatryLock()(line 68) and releases it infinally(line 79). ButsafePurge()also callslock.unlock()(line 154), causing anIllegalMonitorStateExceptionor releasing a lock that a different thread now holds. - Non-thread-safe static
HashMap(line 40) —instancesis a plainHashMapaccessed from multiple threads viacomputeIfAbsent. Should beConcurrentHashMap. - Static mutable
ExecutorProviderbinding — Thevolatilefield +bind()pattern works but is fragile; a null check race exists betweencanLockBeAcquiredcallers andpurgeMessages.
4. MusicManager (music/MusicManager.java)
- Inconsistent synchronization —
isConnected(),connect(),disconnect(),play()aresynchronized, butisCurrentlyPlaying()(line 114),getCurrentlyPlaying()(line 110),stopTrack()(line 138),resume()(line 142) are not. If these are called from event threads concurrently withdisconnect(), they can see stale state.
5. BaseThreadFactory (core/thread/BaseThreadFactory.java)
- Threads are not set as daemon threads, which can prevent clean JVM shutdown if an executor leaks.
Proposed Changes
ThreadManager
- Replace
newCachedThreadPool()withnewFixedThreadPool()or a boundedThreadPoolExecutorwith a reasonable max (e.g.,availableProcessors() * 2). - Fix shutdown loop: wrap each executor's
shutdown()/awaitTermination()in its own try-catch so one failure doesn't skip the rest. - Consider adding a
getExecutorStatus()method for observability.
BaseThreadFactory
- Set
thread.setDaemon(true)to prevent JVM hang on unclean shutdown.
TaskManagerImpl
- Fix
computeIfAbsentnull return: use a sentinel or simply don't store an entry for tasks that don't need locking (useget()+ null check instead). - Remove misleading
synchronizedongetTasks()or make all task mutations consistently synchronized. - Guard
hasTask()with proper synchronization or accept eventual consistency and document it. - Fix lock cleanup race in
handle()— don't remove fromtasksLockinside the lock-held section; letremoveTask()handle cleanup.
ChannelMessagePurger
- Remove the
lock.unlock()call on line 154 insidesafePurge()— thefinallyblock inpurgeMessages()already handles it. - Change
instancesfromHashMaptoConcurrentHashMap.
MusicManager
- Either synchronize all public methods that touch
player/AudioManagerstate, or removesynchronizedfrom methods where JDA already guarantees thread safety and document the threading model.
Delivery Steps
✓ Step 1: Fix ThreadManager and BaseThreadFactory
ThreadManager uses bounded pools and has correct shutdown; BaseThreadFactory produces daemon threads.
- Replace
Executors.newCachedThreadPool()inThreadManager.getExecutor()with a boundedThreadPoolExecutor(core=0, max=availableProcessors*2, 60s keepAlive, LinkedBlockingQueue). - Fix
stop()shutdown loop: wrap each executor'sshutdown()+awaitTermination()in its own try-catch so anInterruptedExceptionon one executor doesn't skip the rest. - In
BaseThreadFactory.newThread(), addthread.setDaemon(true). - Verify the
getScheduledExecutor()pool size is still appropriate.
✓ Step 2: Fix TaskManagerImpl thread-safety bugs
TaskManagerImpl has no null-in-ConcurrentHashMap violations and consistent synchronization.
- Replace the
computeIfAbsentthat returns null (line 178-184) with a pattern that avoids storing null: check task type first, only callcomputeIfAbsentwhen a lock is actually needed. - Remove the misleading
synchronizedkeyword fromgetTasks(), or switch to a consistent locking strategy across all mutation methods. - Fix the lock cleanup race in
handle(): do not remove fromtasksLockat line 193; letremoveTask()be the sole owner of lock cleanup. - Document the threading contract for
hasTask()(check-then-act across two collections is inherently racy).
✓ Step 3: Fix ChannelMessagePurger concurrency bugs
ChannelMessagePurger has no double-unlock and uses thread-safe collections.
- Remove the redundant
lock.unlock()call on line 154 insafePurge()— thefinallyblock inpurgeMessages()(line 79) already releases the lock. - Change the static
instancesfield (line 40) fromHashMaptoConcurrentHashMap. - Remove the
synchronizedkeyword fromcountAllChannelMessages()andsafePurge()since theReentrantLockalready provides mutual exclusion.
✓ Step 4: Harmonize MusicManager synchronization
MusicManager has a consistent and documented threading model.
- Add
synchronizedtoisCurrentlyPlaying(),getCurrentlyPlaying(),stopTrack(), andresume()to match the existing pattern onconnect()/disconnect()/play(). - Alternatively, if JDA's
AudioPlayeris already thread-safe, removesynchronizedfrom all methods and add a class-level Javadoc explaining the threading model. - Investigate which approach is correct by checking Lavaplayer's
AudioPlayerthread-safety guarantees, then apply consistently.