Files
bruno/packages/bruno-electron/tests/app/collection-tree-batcher.spec.js
Chirag Chandrashekhar f76f487211 Performance/file parse and mount (#6975)
* Refactor: optimize collection updates with batch processing

- Introduced BatchAggregator to handle IPC events in batches, reducing Redux dispatch overhead during collection mounting.
- Updated collection watcher to utilize batch processing for adding files and directories, improving UI performance.
- Implemented ParsedFileCacheStore using LMDB for efficient caching of parsed file content, enhancing loading speed and reducing redundant parsing.
- Adjusted collection slice to support batch addition of items, minimizing re-renders and improving state management.
- Updated relevant components to reflect changes in loading states and collection data handling.

* feat: add cache management to preferences

- Introduced a new Cache component in the Preferences section to display cache statistics and allow users to purge the cache.
- Implemented IPC handlers for fetching cache stats and purging the cache in the Electron main process.
- Added styled components for better UI presentation of cache information.
- Updated Preferences component to include a new tab for cache management.

* fix: update package-lock.json to change 'devOptional' to 'dev' for several Babel dependencies

* refactor: update batch aggregation parameters for improved performance

- Increased DISPATCH_INTERVAL_MS from 150ms to 200ms for better timing control.
- Adjusted MAX_BATCH_SIZE from 200 to 300 items to enhance batch processing efficiency.

* feat: enhance collection loading state and improve batch aggregator functionality

- Added isLoading property to collections slice to manage loading state during collection operations.
- Updated getAggregator function calls in collection-watcher to include collectionUid for better context in batch processing.
- Normalized directory path handling in parsed-file-cache to ensure consistent prefix creation for cache keys.

* fix: update loading state and transient file handling in collections slice

- Changed isLoading property to false during collection initialization for accurate loading state representation.
- Introduced isTransient flag for directories and files to differentiate between transient and non-transient items.
- Enhanced logic for handling transient directories and files during collection processing to improve state management.

* feat: add batch processing support for file additions in task middleware

- Implemented a new listener for collectionBatchAddItems to handle batch file additions.
- Enhanced task management by checking for pending OPEN_REQUEST tasks that match added files.
- Improved tab management by dispatching addTab actions for matching files and removing corresponding tasks from the queue.

* feat: enable ASAR packaging and unpacking for LMDB binaries in Electron build configuration

- Added ASAR support to the Electron build configuration for the Bruno application.
- Specified unpacking rules for LMDB native binaries to ensure proper loading during runtime.

* feat: implement parsed file cache using IndexedDB for improved performance

- Introduced a new `parsedFileCacheStore` utilizing IndexedDB for caching parsed file data.
- Replaced the previous LMDB-based cache implementation to enhance performance and reliability.
- Updated IPC handlers to manage cache operations such as get, set, invalidate, and clear.
- Integrated the new cache store into various components, ensuring efficient data retrieval and storage.
- Added pruning functionality to remove outdated cache entries on startup.

* refactor: update collection root and item handling to preserve UIDs

- Modified the way collection roots and folder items are assigned by using `mergeRootWithPreservedUids` and `mergeRequestWithPreservedUids` to ensure UIDs are maintained during updates.
- This change enhances data integrity when managing collections and their associated files.

* refactor: pass mainWindow reference to parsedFileCacheStore initialization

- Updated the `initialize` method in `ParsedFileCacheStore` to accept a `mainWindow` parameter, allowing for direct access to the main window instance in IPC handlers.
- This change improves the handling of IPC requests by ensuring the correct window context is used for sending messages.

* refactor: optimize getStats method in parsedFileCacheStore for performance

- Replaced manual counting of total files with a direct count() call for O(1) performance.
- Updated the collection counting logic to utilize openKeyCursor with 'nextunique' for improved efficiency in counting unique collection paths.
- These changes enhance the performance of the getStats method by reducing the complexity of file and collection counting.

* fix: update key generation in parsedFileCache to use newline separator

- Changed the key generation logic in `generateKey` from a null character to a newline character for improved readability and consistency in cache keys.

* refactor: rename batch-aggregator to collection-tree-batcher and add tests

- Rename BatchAggregator class to CollectionTreeBatcher
- Rename getAggregator/removeAggregator to getBatcher/removeBatcher
- Update imports and variable names in collection-watcher.js
- Add backward-compatible aliases for old names
- Add 22 unit tests covering all functionality

* refactor: update key generation in parsedFileCache to use custom separator

- Changed the key generation logic in `generateKey` to use a custom separator (↝) instead of a newline character for improved readability and consistency in cache keys.

* fix: add missing reject handler and fix directory prefix collision

- Add reject to Promise and pendingRequests in parsed-file-cache-idb.js
- Normalize dirPath with trailing separator in invalidateDirectory to
  prevent false matches (e.g., /foo/bar matching /foo/barley)
- Use platform-specific path.sep for cross-platform compatibility

* fix: add error handling in parsedFileCache and update window close event

- Added a catch block to handle errors in the database promise in parsedFileCache.
- Updated the window close event listener in collection-tree-batcher to use `once` for better resource management.

* fix: add LRU eviction when IndexedDB quota is exceeded

Handle QuotaExceededError in setEntry by automatically evicting
the oldest 20% of cache entries and retrying the write operation.

* fix: use once instead of on in mock window for batcher tests

---------

Co-authored-by: Chirag Chandrashekhar <cchirag85@gmail.com>
2026-02-25 19:15:48 +05:30

313 lines
9.3 KiB
JavaScript

const { CollectionTreeBatcher, getBatcher, removeBatcher } = require('../../src/app/collection-tree-batcher');
// Mock BrowserWindow
const createMockWindow = (id = 1) => {
const listeners = {};
return {
id,
isDestroyed: jest.fn(() => false),
once: jest.fn((event, callback) => {
listeners[event] = callback;
}),
emit: (event) => {
if (listeners[event]) {
listeners[event]();
}
},
webContents: {
send: jest.fn()
}
};
};
describe('CollectionTreeBatcher', () => {
beforeEach(() => {
jest.useFakeTimers();
});
afterEach(() => {
jest.useRealTimers();
});
describe('constructor', () => {
it('should initialize with empty queue and no timer', () => {
const win = createMockWindow();
const batcher = new CollectionTreeBatcher(win, 'collection-1');
expect(batcher.queue).toEqual([]);
expect(batcher.timer).toBeNull();
expect(batcher.isDestroyed).toBe(false);
});
});
describe('add()', () => {
it('should add events to the queue', () => {
const win = createMockWindow();
const batcher = new CollectionTreeBatcher(win, 'collection-1');
batcher.add('addFile', { path: '/test/file.bru' });
expect(batcher.queue).toHaveLength(1);
expect(batcher.queue[0]).toEqual({
eventType: 'addFile',
payload: { path: '/test/file.bru' }
});
});
it('should schedule a flush after adding an event', () => {
const win = createMockWindow();
const batcher = new CollectionTreeBatcher(win, 'collection-1');
batcher.add('addFile', { path: '/test/file.bru' });
expect(batcher.timer).not.toBeNull();
});
it('should not add events if window is destroyed', () => {
const win = createMockWindow();
win.isDestroyed.mockReturnValue(true);
const batcher = new CollectionTreeBatcher(win, 'collection-1');
batcher.add('addFile', { path: '/test/file.bru' });
expect(batcher.queue).toHaveLength(0);
});
it('should not add events if batcher is destroyed', () => {
const win = createMockWindow();
const batcher = new CollectionTreeBatcher(win, 'collection-1');
batcher.destroy();
batcher.add('addFile', { path: '/test/file.bru' });
expect(batcher.queue).toHaveLength(0);
});
});
describe('flush()', () => {
it('should send batch to renderer and clear queue', () => {
const win = createMockWindow();
const batcher = new CollectionTreeBatcher(win, 'collection-1');
batcher.add('addFile', { path: '/test/file1.bru' });
batcher.add('addDir', { path: '/test/folder' });
batcher.flush();
expect(win.webContents.send).toHaveBeenCalledWith('main:collection-tree-batch-updated', [
{ eventType: 'addFile', payload: { path: '/test/file1.bru' } },
{ eventType: 'addDir', payload: { path: '/test/folder' } }
]);
expect(batcher.queue).toHaveLength(0);
});
it('should not send if queue is empty', () => {
const win = createMockWindow();
const batcher = new CollectionTreeBatcher(win, 'collection-1');
batcher.flush();
expect(win.webContents.send).not.toHaveBeenCalled();
});
it('should clear pending timer on flush', () => {
const win = createMockWindow();
const batcher = new CollectionTreeBatcher(win, 'collection-1');
batcher.add('addFile', { path: '/test/file.bru' });
expect(batcher.timer).not.toBeNull();
batcher.flush();
expect(batcher.timer).toBeNull();
});
it('should not send if window is destroyed', () => {
const win = createMockWindow();
const batcher = new CollectionTreeBatcher(win, 'collection-1');
batcher.add('addFile', { path: '/test/file.bru' });
win.isDestroyed.mockReturnValue(true);
batcher.flush();
expect(win.webContents.send).not.toHaveBeenCalled();
});
});
describe('time-based flush', () => {
it('should auto-flush after DISPATCH_INTERVAL_MS (200ms)', () => {
const win = createMockWindow();
const batcher = new CollectionTreeBatcher(win, 'collection-1');
batcher.add('addFile', { path: '/test/file.bru' });
expect(win.webContents.send).not.toHaveBeenCalled();
jest.advanceTimersByTime(200);
expect(win.webContents.send).toHaveBeenCalledWith('main:collection-tree-batch-updated', [
{ eventType: 'addFile', payload: { path: '/test/file.bru' } }
]);
});
it('should not schedule multiple timers', () => {
const win = createMockWindow();
const batcher = new CollectionTreeBatcher(win, 'collection-1');
batcher.add('addFile', { path: '/test/file1.bru' });
const firstTimer = batcher.timer;
batcher.add('addFile', { path: '/test/file2.bru' });
expect(batcher.timer).toBe(firstTimer);
});
});
describe('size-based flush', () => {
it('should auto-flush when reaching MAX_BATCH_SIZE (300)', () => {
const win = createMockWindow();
const batcher = new CollectionTreeBatcher(win, 'collection-1');
// Add 299 events - should not flush
for (let i = 0; i < 299; i++) {
batcher.add('addFile', { path: `/test/file${i}.bru` });
}
expect(win.webContents.send).not.toHaveBeenCalled();
expect(batcher.queue).toHaveLength(299);
// Add 300th event - should trigger flush
batcher.add('addFile', { path: '/test/file299.bru' });
expect(win.webContents.send).toHaveBeenCalledTimes(1);
expect(batcher.queue).toHaveLength(0);
});
});
describe('size()', () => {
it('should return current queue size', () => {
const win = createMockWindow();
const batcher = new CollectionTreeBatcher(win, 'collection-1');
expect(batcher.size()).toBe(0);
batcher.add('addFile', { path: '/test/file1.bru' });
expect(batcher.size()).toBe(1);
batcher.add('addFile', { path: '/test/file2.bru' });
expect(batcher.size()).toBe(2);
});
});
describe('clear()', () => {
it('should clear the queue without sending', () => {
const win = createMockWindow();
const batcher = new CollectionTreeBatcher(win, 'collection-1');
batcher.add('addFile', { path: '/test/file.bru' });
batcher.clear();
expect(batcher.queue).toHaveLength(0);
expect(batcher.timer).toBeNull();
expect(win.webContents.send).not.toHaveBeenCalled();
});
});
describe('destroy()', () => {
it('should mark batcher as destroyed and clear queue', () => {
const win = createMockWindow();
const batcher = new CollectionTreeBatcher(win, 'collection-1');
batcher.add('addFile', { path: '/test/file.bru' });
batcher.destroy();
expect(batcher.isDestroyed).toBe(true);
expect(batcher.queue).toHaveLength(0);
expect(batcher.win).toBeNull();
});
});
describe('error handling', () => {
it('should handle send errors gracefully', () => {
const win = createMockWindow();
win.webContents.send.mockImplementation(() => {
throw new Error('Window closed');
});
const batcher = new CollectionTreeBatcher(win, 'collection-1');
const consoleSpy = jest.spyOn(console, 'error').mockImplementation(() => {});
batcher.add('addFile', { path: '/test/file.bru' });
batcher.flush();
expect(consoleSpy).toHaveBeenCalledWith('CollectionTreeBatcher: Error sending batch:', expect.any(Error));
consoleSpy.mockRestore();
});
});
});
describe('getBatcher / removeBatcher', () => {
beforeEach(() => {
jest.useFakeTimers();
});
afterEach(() => {
jest.useRealTimers();
});
it('should create and return a batcher for a window', () => {
const win = createMockWindow(100);
const batcher = getBatcher(win, 'collection-1');
expect(batcher).toBeInstanceOf(CollectionTreeBatcher);
});
it('should return the same batcher for the same window and collection', () => {
const win = createMockWindow(101);
const batcher1 = getBatcher(win, 'collection-1');
const batcher2 = getBatcher(win, 'collection-1');
expect(batcher1).toBe(batcher2);
});
it('should return different batchers for different collections', () => {
const win = createMockWindow(102);
const batcher1 = getBatcher(win, 'collection-1');
const batcher2 = getBatcher(win, 'collection-2');
expect(batcher1).not.toBe(batcher2);
});
it('should return different batchers for different windows', () => {
const win1 = createMockWindow(103);
const win2 = createMockWindow(104);
const batcher1 = getBatcher(win1, 'collection-1');
const batcher2 = getBatcher(win2, 'collection-1');
expect(batcher1).not.toBe(batcher2);
});
it('should clean up batcher when window is closed', () => {
const win = createMockWindow(105);
const batcher = getBatcher(win, 'collection-1');
batcher.add('addFile', { path: '/test/file.bru' });
// Simulate window close
win.emit('closed');
expect(batcher.isDestroyed).toBe(true);
});
it('should remove batcher with removeBatcher', () => {
const win = createMockWindow(106);
const batcher = getBatcher(win, 'collection-1');
removeBatcher(win, 'collection-1');
expect(batcher.isDestroyed).toBe(true);
// Getting batcher again should create a new one
const newBatcher = getBatcher(win, 'collection-1');
expect(newBatcher).not.toBe(batcher);
});
});