mirror of
https://github.com/usebruno/bruno.git
synced 2026-06-28 07:04:10 +00:00
* Refactor: optimize collection updates with batch processing - Introduced BatchAggregator to handle IPC events in batches, reducing Redux dispatch overhead during collection mounting. - Updated collection watcher to utilize batch processing for adding files and directories, improving UI performance. - Implemented ParsedFileCacheStore using LMDB for efficient caching of parsed file content, enhancing loading speed and reducing redundant parsing. - Adjusted collection slice to support batch addition of items, minimizing re-renders and improving state management. - Updated relevant components to reflect changes in loading states and collection data handling. * feat: add cache management to preferences - Introduced a new Cache component in the Preferences section to display cache statistics and allow users to purge the cache. - Implemented IPC handlers for fetching cache stats and purging the cache in the Electron main process. - Added styled components for better UI presentation of cache information. - Updated Preferences component to include a new tab for cache management. * fix: update package-lock.json to change 'devOptional' to 'dev' for several Babel dependencies * refactor: update batch aggregation parameters for improved performance - Increased DISPATCH_INTERVAL_MS from 150ms to 200ms for better timing control. - Adjusted MAX_BATCH_SIZE from 200 to 300 items to enhance batch processing efficiency. * feat: enhance collection loading state and improve batch aggregator functionality - Added isLoading property to collections slice to manage loading state during collection operations. - Updated getAggregator function calls in collection-watcher to include collectionUid for better context in batch processing. - Normalized directory path handling in parsed-file-cache to ensure consistent prefix creation for cache keys. * fix: update loading state and transient file handling in collections slice - Changed isLoading property to false during collection initialization for accurate loading state representation. - Introduced isTransient flag for directories and files to differentiate between transient and non-transient items. - Enhanced logic for handling transient directories and files during collection processing to improve state management. * feat: add batch processing support for file additions in task middleware - Implemented a new listener for collectionBatchAddItems to handle batch file additions. - Enhanced task management by checking for pending OPEN_REQUEST tasks that match added files. - Improved tab management by dispatching addTab actions for matching files and removing corresponding tasks from the queue. * feat: enable ASAR packaging and unpacking for LMDB binaries in Electron build configuration - Added ASAR support to the Electron build configuration for the Bruno application. - Specified unpacking rules for LMDB native binaries to ensure proper loading during runtime. * feat: implement parsed file cache using IndexedDB for improved performance - Introduced a new `parsedFileCacheStore` utilizing IndexedDB for caching parsed file data. - Replaced the previous LMDB-based cache implementation to enhance performance and reliability. - Updated IPC handlers to manage cache operations such as get, set, invalidate, and clear. - Integrated the new cache store into various components, ensuring efficient data retrieval and storage. - Added pruning functionality to remove outdated cache entries on startup. * refactor: update collection root and item handling to preserve UIDs - Modified the way collection roots and folder items are assigned by using `mergeRootWithPreservedUids` and `mergeRequestWithPreservedUids` to ensure UIDs are maintained during updates. - This change enhances data integrity when managing collections and their associated files. * refactor: pass mainWindow reference to parsedFileCacheStore initialization - Updated the `initialize` method in `ParsedFileCacheStore` to accept a `mainWindow` parameter, allowing for direct access to the main window instance in IPC handlers. - This change improves the handling of IPC requests by ensuring the correct window context is used for sending messages. * refactor: optimize getStats method in parsedFileCacheStore for performance - Replaced manual counting of total files with a direct count() call for O(1) performance. - Updated the collection counting logic to utilize openKeyCursor with 'nextunique' for improved efficiency in counting unique collection paths. - These changes enhance the performance of the getStats method by reducing the complexity of file and collection counting. * fix: update key generation in parsedFileCache to use newline separator - Changed the key generation logic in `generateKey` from a null character to a newline character for improved readability and consistency in cache keys. * refactor: rename batch-aggregator to collection-tree-batcher and add tests - Rename BatchAggregator class to CollectionTreeBatcher - Rename getAggregator/removeAggregator to getBatcher/removeBatcher - Update imports and variable names in collection-watcher.js - Add backward-compatible aliases for old names - Add 22 unit tests covering all functionality * refactor: update key generation in parsedFileCache to use custom separator - Changed the key generation logic in `generateKey` to use a custom separator (↝) instead of a newline character for improved readability and consistency in cache keys. * fix: add missing reject handler and fix directory prefix collision - Add reject to Promise and pendingRequests in parsed-file-cache-idb.js - Normalize dirPath with trailing separator in invalidateDirectory to prevent false matches (e.g., /foo/bar matching /foo/barley) - Use platform-specific path.sep for cross-platform compatibility * fix: add error handling in parsedFileCache and update window close event - Added a catch block to handle errors in the database promise in parsedFileCache. - Updated the window close event listener in collection-tree-batcher to use `once` for better resource management. * fix: add LRU eviction when IndexedDB quota is exceeded Handle QuotaExceededError in setEntry by automatically evicting the oldest 20% of cache entries and retrying the write operation. * fix: use once instead of on in mock window for batcher tests --------- Co-authored-by: Chirag Chandrashekhar <cchirag85@gmail.com>
313 lines
9.3 KiB
JavaScript
313 lines
9.3 KiB
JavaScript
const { CollectionTreeBatcher, getBatcher, removeBatcher } = require('../../src/app/collection-tree-batcher');
|
|
|
|
// Mock BrowserWindow
|
|
const createMockWindow = (id = 1) => {
|
|
const listeners = {};
|
|
return {
|
|
id,
|
|
isDestroyed: jest.fn(() => false),
|
|
once: jest.fn((event, callback) => {
|
|
listeners[event] = callback;
|
|
}),
|
|
emit: (event) => {
|
|
if (listeners[event]) {
|
|
listeners[event]();
|
|
}
|
|
},
|
|
webContents: {
|
|
send: jest.fn()
|
|
}
|
|
};
|
|
};
|
|
|
|
describe('CollectionTreeBatcher', () => {
|
|
beforeEach(() => {
|
|
jest.useFakeTimers();
|
|
});
|
|
|
|
afterEach(() => {
|
|
jest.useRealTimers();
|
|
});
|
|
|
|
describe('constructor', () => {
|
|
it('should initialize with empty queue and no timer', () => {
|
|
const win = createMockWindow();
|
|
const batcher = new CollectionTreeBatcher(win, 'collection-1');
|
|
|
|
expect(batcher.queue).toEqual([]);
|
|
expect(batcher.timer).toBeNull();
|
|
expect(batcher.isDestroyed).toBe(false);
|
|
});
|
|
});
|
|
|
|
describe('add()', () => {
|
|
it('should add events to the queue', () => {
|
|
const win = createMockWindow();
|
|
const batcher = new CollectionTreeBatcher(win, 'collection-1');
|
|
|
|
batcher.add('addFile', { path: '/test/file.bru' });
|
|
|
|
expect(batcher.queue).toHaveLength(1);
|
|
expect(batcher.queue[0]).toEqual({
|
|
eventType: 'addFile',
|
|
payload: { path: '/test/file.bru' }
|
|
});
|
|
});
|
|
|
|
it('should schedule a flush after adding an event', () => {
|
|
const win = createMockWindow();
|
|
const batcher = new CollectionTreeBatcher(win, 'collection-1');
|
|
|
|
batcher.add('addFile', { path: '/test/file.bru' });
|
|
|
|
expect(batcher.timer).not.toBeNull();
|
|
});
|
|
|
|
it('should not add events if window is destroyed', () => {
|
|
const win = createMockWindow();
|
|
win.isDestroyed.mockReturnValue(true);
|
|
const batcher = new CollectionTreeBatcher(win, 'collection-1');
|
|
|
|
batcher.add('addFile', { path: '/test/file.bru' });
|
|
|
|
expect(batcher.queue).toHaveLength(0);
|
|
});
|
|
|
|
it('should not add events if batcher is destroyed', () => {
|
|
const win = createMockWindow();
|
|
const batcher = new CollectionTreeBatcher(win, 'collection-1');
|
|
batcher.destroy();
|
|
|
|
batcher.add('addFile', { path: '/test/file.bru' });
|
|
|
|
expect(batcher.queue).toHaveLength(0);
|
|
});
|
|
});
|
|
|
|
describe('flush()', () => {
|
|
it('should send batch to renderer and clear queue', () => {
|
|
const win = createMockWindow();
|
|
const batcher = new CollectionTreeBatcher(win, 'collection-1');
|
|
|
|
batcher.add('addFile', { path: '/test/file1.bru' });
|
|
batcher.add('addDir', { path: '/test/folder' });
|
|
|
|
batcher.flush();
|
|
|
|
expect(win.webContents.send).toHaveBeenCalledWith('main:collection-tree-batch-updated', [
|
|
{ eventType: 'addFile', payload: { path: '/test/file1.bru' } },
|
|
{ eventType: 'addDir', payload: { path: '/test/folder' } }
|
|
]);
|
|
expect(batcher.queue).toHaveLength(0);
|
|
});
|
|
|
|
it('should not send if queue is empty', () => {
|
|
const win = createMockWindow();
|
|
const batcher = new CollectionTreeBatcher(win, 'collection-1');
|
|
|
|
batcher.flush();
|
|
|
|
expect(win.webContents.send).not.toHaveBeenCalled();
|
|
});
|
|
|
|
it('should clear pending timer on flush', () => {
|
|
const win = createMockWindow();
|
|
const batcher = new CollectionTreeBatcher(win, 'collection-1');
|
|
|
|
batcher.add('addFile', { path: '/test/file.bru' });
|
|
expect(batcher.timer).not.toBeNull();
|
|
|
|
batcher.flush();
|
|
expect(batcher.timer).toBeNull();
|
|
});
|
|
|
|
it('should not send if window is destroyed', () => {
|
|
const win = createMockWindow();
|
|
const batcher = new CollectionTreeBatcher(win, 'collection-1');
|
|
|
|
batcher.add('addFile', { path: '/test/file.bru' });
|
|
win.isDestroyed.mockReturnValue(true);
|
|
|
|
batcher.flush();
|
|
|
|
expect(win.webContents.send).not.toHaveBeenCalled();
|
|
});
|
|
});
|
|
|
|
describe('time-based flush', () => {
|
|
it('should auto-flush after DISPATCH_INTERVAL_MS (200ms)', () => {
|
|
const win = createMockWindow();
|
|
const batcher = new CollectionTreeBatcher(win, 'collection-1');
|
|
|
|
batcher.add('addFile', { path: '/test/file.bru' });
|
|
|
|
expect(win.webContents.send).not.toHaveBeenCalled();
|
|
|
|
jest.advanceTimersByTime(200);
|
|
|
|
expect(win.webContents.send).toHaveBeenCalledWith('main:collection-tree-batch-updated', [
|
|
{ eventType: 'addFile', payload: { path: '/test/file.bru' } }
|
|
]);
|
|
});
|
|
|
|
it('should not schedule multiple timers', () => {
|
|
const win = createMockWindow();
|
|
const batcher = new CollectionTreeBatcher(win, 'collection-1');
|
|
|
|
batcher.add('addFile', { path: '/test/file1.bru' });
|
|
const firstTimer = batcher.timer;
|
|
|
|
batcher.add('addFile', { path: '/test/file2.bru' });
|
|
|
|
expect(batcher.timer).toBe(firstTimer);
|
|
});
|
|
});
|
|
|
|
describe('size-based flush', () => {
|
|
it('should auto-flush when reaching MAX_BATCH_SIZE (300)', () => {
|
|
const win = createMockWindow();
|
|
const batcher = new CollectionTreeBatcher(win, 'collection-1');
|
|
|
|
// Add 299 events - should not flush
|
|
for (let i = 0; i < 299; i++) {
|
|
batcher.add('addFile', { path: `/test/file${i}.bru` });
|
|
}
|
|
expect(win.webContents.send).not.toHaveBeenCalled();
|
|
expect(batcher.queue).toHaveLength(299);
|
|
|
|
// Add 300th event - should trigger flush
|
|
batcher.add('addFile', { path: '/test/file299.bru' });
|
|
|
|
expect(win.webContents.send).toHaveBeenCalledTimes(1);
|
|
expect(batcher.queue).toHaveLength(0);
|
|
});
|
|
});
|
|
|
|
describe('size()', () => {
|
|
it('should return current queue size', () => {
|
|
const win = createMockWindow();
|
|
const batcher = new CollectionTreeBatcher(win, 'collection-1');
|
|
|
|
expect(batcher.size()).toBe(0);
|
|
|
|
batcher.add('addFile', { path: '/test/file1.bru' });
|
|
expect(batcher.size()).toBe(1);
|
|
|
|
batcher.add('addFile', { path: '/test/file2.bru' });
|
|
expect(batcher.size()).toBe(2);
|
|
});
|
|
});
|
|
|
|
describe('clear()', () => {
|
|
it('should clear the queue without sending', () => {
|
|
const win = createMockWindow();
|
|
const batcher = new CollectionTreeBatcher(win, 'collection-1');
|
|
|
|
batcher.add('addFile', { path: '/test/file.bru' });
|
|
batcher.clear();
|
|
|
|
expect(batcher.queue).toHaveLength(0);
|
|
expect(batcher.timer).toBeNull();
|
|
expect(win.webContents.send).not.toHaveBeenCalled();
|
|
});
|
|
});
|
|
|
|
describe('destroy()', () => {
|
|
it('should mark batcher as destroyed and clear queue', () => {
|
|
const win = createMockWindow();
|
|
const batcher = new CollectionTreeBatcher(win, 'collection-1');
|
|
|
|
batcher.add('addFile', { path: '/test/file.bru' });
|
|
batcher.destroy();
|
|
|
|
expect(batcher.isDestroyed).toBe(true);
|
|
expect(batcher.queue).toHaveLength(0);
|
|
expect(batcher.win).toBeNull();
|
|
});
|
|
});
|
|
|
|
describe('error handling', () => {
|
|
it('should handle send errors gracefully', () => {
|
|
const win = createMockWindow();
|
|
win.webContents.send.mockImplementation(() => {
|
|
throw new Error('Window closed');
|
|
});
|
|
const batcher = new CollectionTreeBatcher(win, 'collection-1');
|
|
const consoleSpy = jest.spyOn(console, 'error').mockImplementation(() => {});
|
|
|
|
batcher.add('addFile', { path: '/test/file.bru' });
|
|
batcher.flush();
|
|
|
|
expect(consoleSpy).toHaveBeenCalledWith('CollectionTreeBatcher: Error sending batch:', expect.any(Error));
|
|
consoleSpy.mockRestore();
|
|
});
|
|
});
|
|
});
|
|
|
|
describe('getBatcher / removeBatcher', () => {
|
|
beforeEach(() => {
|
|
jest.useFakeTimers();
|
|
});
|
|
|
|
afterEach(() => {
|
|
jest.useRealTimers();
|
|
});
|
|
|
|
it('should create and return a batcher for a window', () => {
|
|
const win = createMockWindow(100);
|
|
const batcher = getBatcher(win, 'collection-1');
|
|
|
|
expect(batcher).toBeInstanceOf(CollectionTreeBatcher);
|
|
});
|
|
|
|
it('should return the same batcher for the same window and collection', () => {
|
|
const win = createMockWindow(101);
|
|
const batcher1 = getBatcher(win, 'collection-1');
|
|
const batcher2 = getBatcher(win, 'collection-1');
|
|
|
|
expect(batcher1).toBe(batcher2);
|
|
});
|
|
|
|
it('should return different batchers for different collections', () => {
|
|
const win = createMockWindow(102);
|
|
const batcher1 = getBatcher(win, 'collection-1');
|
|
const batcher2 = getBatcher(win, 'collection-2');
|
|
|
|
expect(batcher1).not.toBe(batcher2);
|
|
});
|
|
|
|
it('should return different batchers for different windows', () => {
|
|
const win1 = createMockWindow(103);
|
|
const win2 = createMockWindow(104);
|
|
const batcher1 = getBatcher(win1, 'collection-1');
|
|
const batcher2 = getBatcher(win2, 'collection-1');
|
|
|
|
expect(batcher1).not.toBe(batcher2);
|
|
});
|
|
|
|
it('should clean up batcher when window is closed', () => {
|
|
const win = createMockWindow(105);
|
|
const batcher = getBatcher(win, 'collection-1');
|
|
|
|
batcher.add('addFile', { path: '/test/file.bru' });
|
|
|
|
// Simulate window close
|
|
win.emit('closed');
|
|
|
|
expect(batcher.isDestroyed).toBe(true);
|
|
});
|
|
|
|
it('should remove batcher with removeBatcher', () => {
|
|
const win = createMockWindow(106);
|
|
const batcher = getBatcher(win, 'collection-1');
|
|
|
|
removeBatcher(win, 'collection-1');
|
|
|
|
expect(batcher.isDestroyed).toBe(true);
|
|
|
|
// Getting batcher again should create a new one
|
|
const newBatcher = getBatcher(win, 'collection-1');
|
|
expect(newBatcher).not.toBe(batcher);
|
|
});
|
|
});
|