* Feature request network graph

* Fixing PR442:

- Fixed:
    - UI design elemets
    - UI and button colors
    - JSON export
    - recent activity is default again
- Removed:
    - Online/Offline UI labels
    - left-click menu on hosts
- Added:
    - small pulsing dot inside the hosts to indicate online status like in the left bar

* fix: electron build errors and skip macos job

* fix: testflight submit failure

* fix: made submit job match build type

* fix: resolve Vite build warnings for mixed static/dynamic imports (#473)

* Update Crowdin configuration file

* Update Crowdin configuration file

* fix: resolve Vite build warnings for mixed static/dynamic imports

- Convert all dynamic imports of main-axios.ts to static imports (10 files)
- Convert all dynamic imports of sonner to static imports (4 files)
- Add manual chunking configuration to vite.config.ts for better bundle splitting
  - react-vendor: React and React DOM
  - ui-vendor: Radix UI, lucide-react, clsx, tailwind-merge
  - monaco: Monaco Editor
  - codemirror: CodeMirror and related packages
- Increase chunkSizeWarningLimit to 1000kB

This resolves Vite warnings about mixed import strategies preventing
proper code-splitting.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: Luke Gustafson <88517757+LukeGus@users.noreply.github.com>
Co-authored-by: Termix CI <ci@termix.dev>
Co-authored-by: Claude <noreply@anthropic.com>

* fix: file manager incorrectly decoding/encoding when editing files (made base64/utf8 dependent)

* fix: build error on docker

* Handle enter button (#481)

* Update Crowdin configuration file

* Update Crowdin configuration file

* Update Linux Portable section with AUR link (#474)

* fix: file manager incorrectly decoding/encoding when editing files (#476)

* fix: electron build errors and skip macos job

* fix: testflight submit failure

* fix: made submit job match build type

* fix: resolve Vite build warnings for mixed static/dynamic imports (#473)

* Update Crowdin configuration file

* Update Crowdin configuration file

* fix: resolve Vite build warnings for mixed static/dynamic imports

- Convert all dynamic imports of main-axios.ts to static imports (10 files)
- Convert all dynamic imports of sonner to static imports (4 files)
- Add manual chunking configuration to vite.config.ts for better bundle splitting
  - react-vendor: React and React DOM
  - ui-vendor: Radix UI, lucide-react, clsx, tailwind-merge
  - monaco: Monaco Editor
  - codemirror: CodeMirror and related packages
- Increase chunkSizeWarningLimit to 1000kB

This resolves Vite warnings about mixed import strategies preventing
proper code-splitting.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: Luke Gustafson <88517757+LukeGus@users.noreply.github.com>
Co-authored-by: Termix CI <ci@termix.dev>
Co-authored-by: Claude <noreply@anthropic.com>

* fix: file manager incorrectly decoding/encoding when editing files (made base64/utf8 dependent)

---------

Co-authored-by: Jefferson Nunn <89030989+jeffersonwarrior@users.noreply.github.com>
Co-authored-by: Termix CI <ci@termix.dev>
Co-authored-by: Claude <noreply@anthropic.com>

* fix: build error on docker (#477)

* fix: electron build errors and skip macos job

* fix: testflight submit failure

* fix: made submit job match build type

* fix: resolve Vite build warnings for mixed static/dynamic imports (#473)

* Update Crowdin configuration file

* Update Crowdin configuration file

* fix: resolve Vite build warnings for mixed static/dynamic imports

- Convert all dynamic imports of main-axios.ts to static imports (10 files)
- Convert all dynamic imports of sonner to static imports (4 files)
- Add manual chunking configuration to vite.config.ts for better bundle splitting
  - react-vendor: React and React DOM
  - ui-vendor: Radix UI, lucide-react, clsx, tailwind-merge
  - monaco: Monaco Editor
  - codemirror: CodeMirror and related packages
- Increase chunkSizeWarningLimit to 1000kB

This resolves Vite warnings about mixed import strategies preventing
proper code-splitting.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: Luke Gustafson <88517757+LukeGus@users.noreply.github.com>
Co-authored-by: Termix CI <ci@termix.dev>
Co-authored-by: Claude <noreply@anthropic.com>

* fix: file manager incorrectly decoding/encoding when editing files (made base64/utf8 dependent)

* fix: build error on docker

---------

Co-authored-by: Jefferson Nunn <89030989+jeffersonwarrior@users.noreply.github.com>
Co-authored-by: Termix CI <ci@termix.dev>
Co-authored-by: Claude <noreply@anthropic.com>

* Increase max old space size for npm builds

* Increase Node.js memory limit in Dockerfile

* Remove NODE_OPTIONS from build commands in Dockerfile

* Change runner to blacksmith-4vcpu-ubuntu-2404

* fix: build error on docker

* Add handle on enter button;

---------

Co-authored-by: Luke Gustafson <88517757+LukeGus@users.noreply.github.com>
Co-authored-by: Gaylord Julien <g.j@mailbox.org>
Co-authored-by: Jefferson Nunn <89030989+jeffersonwarrior@users.noreply.github.com>
Co-authored-by: Termix CI <ci@termix.dev>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: LukeGus <bugattiguy527@gmail.com>

* fix: remove top tech

* fix: update readme

* fix: prevent long container names from overflowing card (#496)

Added min-w-0 to CardTitle to allow text truncation in flexbox.
Without this, flex items have min-width: auto which prevents
the truncate class from working properly.

Fixes #411

* fix: use SFTP readdir for file listing to support non-Linux systems (#495)

The file manager now uses SFTP readdir as the primary method for
listing files, with ls -la as a fallback. This enables compatibility
with MikroTik RouterOS and other non-Linux systems that don't have
standard shell commands.

Fixes #317

* fix: restore SSH connection timeout to 120s for 2FA authentication (#494)

The timeout was reduced from 120s to 30s in v1.10, causing 2FA login
failures. Users with keyboard-interactive authentication (TOTP/2FA)
need sufficient time to enter their verification codes before the
SSH connection times out.

Fixes #404

* feat: add Docker container healthcheck (#493)

* fix: owner should not be marked as shared when host is shared to their role (#492)

* fix: use correct MIME types for image preview (#491)

* fix: prevent session reset when updating host properties (#490)

* fix: add shell creation timeout and improve error handling (#489)

* fix: set default lineHeight to 1.0 for TUI apps compatibility (#488)

* fix: delete all related data when removing user (#487)

* fix: nginx permission denied on restricted kernels (#486)

* fix: skip existing hosts and credentials during JSON import (#485)

Added duplicate detection for SSH hosts (by ip+port+username) and
credentials (by name) during import. Existing items are now skipped
by default, or updated if replaceExisting option is enabled.

This matches the existing behavior of importDismissedAlerts.

Fixes #389

* feat: add firewall status widget for server stats (#484)

* Feature: PWA (#479)

* feat: add PWA support with offline capabilities

- Add web app manifest with icons and theme configuration
- Add service worker with cache-first strategy for static assets
- Add useServiceWorker hook for SW registration
- Add PWA meta tags and Apple-specific tags to index.html
- Update vite.config.ts for optimal asset caching

* Update package-lock.json

* New Crowdin updates (#472)

* New translations en.json (Romanian)

* New translations en.json (French)

* New translations en.json (Spanish)

* New translations en.json (Afrikaans)

* New translations en.json (Arabic)

* New translations en.json (Catalan)

* New translations en.json (Czech)

* New translations en.json (Danish)

* New translations en.json (German)

* New translations en.json (Greek)

* New translations en.json (Finnish)

* New translations en.json (Hebrew)

* New translations en.json (Hungarian)

* New translations en.json (Italian)

* New translations en.json (Japanese)

* New translations en.json (Korean)

* New translations en.json (Dutch)

* New translations en.json (Norwegian)

* New translations en.json (Polish)

* New translations en.json (Portuguese)

* New translations en.json (Russian)

* New translations en.json (Serbian (Cyrillic))

* New translations en.json (Swedish)

* New translations en.json (Turkish)

* New translations en.json (Ukrainian)

* New translations en.json (Chinese Simplified)

* New translations en.json (English)

* New translations en.json (Vietnamese)

* New translations en.json (German)

* feat: add listening ports widget for server stats (#483)

Co-authored-by: Luke Gustafson <88517757+LukeGus@users.noreply.github.com>

* feat: fix network stats merge and add openapi jsdocs comments

* feat: add workflow/config to auto generate openapi json

* feat: remove locales

* feat: support URL routes to open terminal directly (#156) (#503)

* fix: resolve merge conflict artifacts in dev-1.10.1

- Fix missing closing tags in AppView.tsx NetworkGraphView
- Fix incomplete catch blocks in server-stats.ts and db/index.ts
- Fix missing closing brace in en.json ports section
- Fix HostManagerApp.tsx import path
- Fix stats-widgets.ts type definition
- Fix schema.ts networkTopology table definition
- Add type annotations in user-data-import.ts

* feat: support URL routes to open terminal directly (#156)

- Add /terminal/{hostNameOrId} route for new format
- Keep /hosts/{id}/terminal for backward compatibility
- Smart detection: numeric IDs for ID lookup, otherwise name lookup
- Clean URL after opening to prevent duplicate on refresh
- Show toast error when host not found

---------

Co-authored-by: Luke Gustafson <88517757+LukeGus@users.noreply.github.com>

* feat: add Ctrl+Alt key remapping for browser-blocked shortcuts (#501)

Browsers intercept Ctrl+W/T/N/Q, making them unusable in terminal.
This adds Ctrl+Alt+<key> as an alternative that sends Ctrl+<key>.

- Ctrl+Alt+W → Ctrl+W (nano search, delete word)
- Ctrl+Alt+T → Ctrl+T (transpose chars)
- Ctrl+Alt+N → Ctrl+N (next line)
- Ctrl+Alt+Q → Ctrl+Q (XON flow control)

Fixes Termix-SSH/Support#407

* feat: remove locales

* New Crowdin updates (#504)

* New translations en.json (Romanian)

* New translations en.json (French)

* New translations en.json (Spanish)

* New translations en.json (Afrikaans)

* New translations en.json (Arabic)

* New translations en.json (Catalan)

* New translations en.json (Czech)

* New translations en.json (Danish)

* New translations en.json (German)

* New translations en.json (Greek)

* New translations en.json (Finnish)

* New translations en.json (Hebrew)

* New translations en.json (Hungarian)

* New translations en.json (Italian)

* New translations en.json (Japanese)

* New translations en.json (Korean)

* New translations en.json (Dutch)

* New translations en.json (Norwegian)

* New translations en.json (Polish)

* New translations en.json (Portuguese)

* New translations en.json (Russian)

* New translations en.json (Serbian (Cyrillic))

* New translations en.json (Swedish)

* New translations en.json (Turkish)

* New translations en.json (Ukrainian)

* New translations en.json (Chinese Simplified)

* New translations en.json (English)

* New translations en.json (Vietnamese)

* New translations en.json (German)

* New translations en.json (Norwegian)

* New translations en.json (Romanian)

* New translations en.json (French)

* New translations en.json (Spanish)

* New translations en.json (Arabic)

* New translations en.json (Czech)

* New translations en.json (Danish)

* New translations en.json (Greek)

* New translations en.json (Finnish)

* New translations en.json (Italian)

* New translations en.json (Japanese)

* New translations en.json (Dutch)

* New translations en.json (Norwegian)

* New translations en.json (Polish)

* New translations en.json (Portuguese)

* New translations en.json (Russian)

* New translations en.json (Swedish)

* New translations en.json (Ukrainian)

* New translations en.json (Chinese Simplified)

* New translations en.json (Chinese Traditional)

* New translations en.json (Finnish)

* New translations en.json (Chinese Simplified)

* New translations en.json (Chinese Traditional)

* feat: add option to disable update checker (#502)

* feat: add option to disable update checker

Add a new setting in User Profile > Settings to disable automatic
update checking on startup and dashboard.

- Adds 'Disable Update Check' toggle in profile settings
- Skips GitHub API calls when disabled (reduces network requests)
- Works for both web app and Electron client

Fixes Termix-SSH/Support#410

* feat: remove locales

---------

Co-authored-by: LukeGus <bugattiguy527@gmail.com>

* New Crowdin updates (#505)

* New translations en.json (Romanian)

* New translations en.json (French)

* New translations en.json (Spanish)

* New translations en.json (Afrikaans)

* New translations en.json (Arabic)

* New translations en.json (Catalan)

* New translations en.json (Czech)

* New translations en.json (Danish)

* New translations en.json (German)

* New translations en.json (Greek)

* New translations en.json (Finnish)

* New translations en.json (Hebrew)

* New translations en.json (Hungarian)

* New translations en.json (Italian)

* New translations en.json (Japanese)

* New translations en.json (Korean)

* New translations en.json (Dutch)

* New translations en.json (Norwegian)

* New translations en.json (Polish)

* New translations en.json (Portuguese)

* New translations en.json (Russian)

* New translations en.json (Serbian (Cyrillic))

* New translations en.json (Swedish)

* New translations en.json (Turkish)

* New translations en.json (Ukrainian)

* New translations en.json (Chinese Simplified)

* New translations en.json (English)

* New translations en.json (Vietnamese)

* New translations en.json (German)

* New translations en.json (Norwegian)

* New translations en.json (Romanian)

* New translations en.json (French)

* New translations en.json (Spanish)

* New translations en.json (Arabic)

* New translations en.json (Czech)

* New translations en.json (Danish)

* New translations en.json (Greek)

* New translations en.json (Finnish)

* New translations en.json (Italian)

* New translations en.json (Japanese)

* New translations en.json (Dutch)

* New translations en.json (Norwegian)

* New translations en.json (Polish)

* New translations en.json (Portuguese)

* New translations en.json (Russian)

* New translations en.json (Swedish)

* New translations en.json (Ukrainian)

* New translations en.json (Chinese Simplified)

* New translations en.json (Chinese Traditional)

* New translations en.json (Finnish)

* New translations en.json (Chinese Simplified)

* New translations en.json (Chinese Traditional)

* New translations en.json (Chinese Simplified)

* New translations en.json (Chinese Traditional)

* New translations en.json (Bulgarian)

* New translations en.json (Indonesian)

* New translations en.json (Hindi)

* feat: add crowdin i18n

* feat: remove locales

* New Crowdin updates (#506)

* New translations en.json (Romanian)

* New translations en.json (French)

* New translations en.json (Spanish)

* New translations en.json (Afrikaans)

* New translations en.json (Arabic)

* New translations en.json (Catalan)

* New translations en.json (Czech)

* New translations en.json (Danish)

* New translations en.json (German)

* New translations en.json (Greek)

* New translations en.json (Finnish)

* New translations en.json (Hebrew)

* New translations en.json (Hungarian)

* New translations en.json (Italian)

* New translations en.json (Japanese)

* New translations en.json (Korean)

* New translations en.json (Dutch)

* New translations en.json (Norwegian)

* New translations en.json (Polish)

* New translations en.json (Portuguese)

* New translations en.json (Russian)

* New translations en.json (Serbian (Cyrillic))

* New translations en.json (Swedish)

* New translations en.json (Turkish)

* New translations en.json (Ukrainian)

* New translations en.json (Chinese Simplified)

* New translations en.json (English)

* New translations en.json (Vietnamese)

* New translations en.json (German)

* New translations en.json (Norwegian)

* New translations en.json (Romanian)

* New translations en.json (French)

* New translations en.json (Spanish)

* New translations en.json (Arabic)

* New translations en.json (Czech)

* New translations en.json (Danish)

* New translations en.json (Greek)

* New translations en.json (Finnish)

* New translations en.json (Italian)

* New translations en.json (Japanese)

* New translations en.json (Dutch)

* New translations en.json (Norwegian)

* New translations en.json (Polish)

* New translations en.json (Portuguese)

* New translations en.json (Russian)

* New translations en.json (Swedish)

* New translations en.json (Ukrainian)

* New translations en.json (Chinese Simplified)

* New translations en.json (Chinese Traditional)

* New translations en.json (Finnish)

* New translations en.json (Chinese Simplified)

* New translations en.json (Chinese Traditional)

* New translations en.json (Chinese Simplified)

* New translations en.json (Chinese Traditional)

* New translations en.json (Bulgarian)

* New translations en.json (Indonesian)

* New translations en.json (Hindi)

* New translations en.json (Romanian)

* New translations en.json (French)

* New translations en.json (Spanish)

* New translations en.json (Afrikaans)

* New translations en.json (Arabic)

* New translations en.json (Catalan)

* New translations en.json (Czech)

* New translations en.json (German)

* New translations en.json (Greek)

* New translations en.json (Finnish)

* New translations en.json (Hebrew)

* New translations en.json (Hungarian)

* New translations en.json (Italian)

* New translations en.json (Japanese)

* New translations en.json (Korean)

* New translations en.json (Dutch)

* New translations en.json (Polish)

* New translations en.json (Portuguese)

* New translations en.json (Russian)

* New translations en.json (Serbian (Cyrillic))

* New translations en.json (Turkish)

* New translations en.json (Ukrainian)

* New translations en.json (Chinese Simplified)

* New translations en.json (Chinese Traditional)

* New translations en.json (Vietnamese)

* New translations en.json (Portuguese, Brazilian)

* New translations en.json (Bulgarian)

* New translations en.json (Indonesian)

* New translations en.json (Hindi)

* New translations en.json (Bengali)

* New translations en.json (Thai)

* feat: update readme

* feat: update readme

* feat: update credential editor to use submitting system and add health monitor

* feat: added toggle for command pallete

* feat: added close button on tab dropdown

* feat: added sidebar management and improved some host manager UI/UX

* feat: re-added missing users.ts route from merge

* feat: add toggle for password reset feature in admin settings (#508)

* feat: add sudo support for file manager operations (#509)

* fix: add sudo support for listFiles and improve permission error handling (#512)

* feat: add sudo support for file manager operations

* fix: add sudo support for listFiles and improve permission error handling

---------

Co-authored-by: Luke Gustafson <88517757+LukeGus@users.noreply.github.com>

* feat: fix sudo password dialog ui, add totp/pass reset limiting, and refreshed users screen when auth is outdated

* feat: add copy password button and fixed new line carriage issues and backend crash for auth key

* feat: added quick connection system (ad-hoc)

* Enter Key for Quick Login (#513)

* feat: added -r and -l support for tunnels

* feat: begin dashboard overhaul by splitting into cards and adding customization

* feat: improved full screen apps, overhauled dashboard, updated server stats ui, etc.

* feat: add auth.tsx suppot for fullscreen

* feat: greatly improve network graph ui/ux and migrated to use translations and theme system

* feat: update to use blacksmith

* feat: improve ui for customized tabs and hide add/edit host/credential when submiting

* feat: add warpgate support with a dialog (terminal only)

* feat: expand warpgate to docker/file manager

* fix: docker not working wtih warpgate and none auth failing for terminal

* fix: prevent owner permission loss when sharing host to own role (#514)

* Update Crowdin configuration file

* Update Crowdin configuration file

* Update Linux Portable section with AUR link (#474)

* fix: file manager incorrectly decoding/encoding when editing files (#476)

* fix: electron build errors and skip macos job

* fix: testflight submit failure

* fix: made submit job match build type

* fix: resolve Vite build warnings for mixed static/dynamic imports (#473)

* Update Crowdin configuration file

* Update Crowdin configuration file

* fix: resolve Vite build warnings for mixed static/dynamic imports

- Convert all dynamic imports of main-axios.ts to static imports (10 files)
- Convert all dynamic imports of sonner to static imports (4 files)
- Add manual chunking configuration to vite.config.ts for better bundle splitting
  - react-vendor: React and React DOM
  - ui-vendor: Radix UI, lucide-react, clsx, tailwind-merge
  - monaco: Monaco Editor
  - codemirror: CodeMirror and related packages
- Increase chunkSizeWarningLimit to 1000kB

This resolves Vite warnings about mixed import strategies preventing
proper code-splitting.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: Luke Gustafson <88517757+LukeGus@users.noreply.github.com>
Co-authored-by: Termix CI <ci@termix.dev>
Co-authored-by: Claude <noreply@anthropic.com>

* fix: file manager incorrectly decoding/encoding when editing files (made base64/utf8 dependent)

---------

Co-authored-by: Jefferson Nunn <89030989+jeffersonwarrior@users.noreply.github.com>
Co-authored-by: Termix CI <ci@termix.dev>
Co-authored-by: Claude <noreply@anthropic.com>

* fix: build error on docker (#477)

* fix: electron build errors and skip macos job

* fix: testflight submit failure

* fix: made submit job match build type

* fix: resolve Vite build warnings for mixed static/dynamic imports (#473)

* Update Crowdin configuration file

* Update Crowdin configuration file

* fix: resolve Vite build warnings for mixed static/dynamic imports

- Convert all dynamic imports of main-axios.ts to static imports (10 files)
- Convert all dynamic imports of sonner to static imports (4 files)
- Add manual chunking configuration to vite.config.ts for better bundle splitting
  - react-vendor: React and React DOM
  - ui-vendor: Radix UI, lucide-react, clsx, tailwind-merge
  - monaco: Monaco Editor
  - codemirror: CodeMirror and related packages
- Increase chunkSizeWarningLimit to 1000kB

This resolves Vite warnings about mixed import strategies preventing
proper code-splitting.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: Luke Gustafson <88517757+LukeGus@users.noreply.github.com>
Co-authored-by: Termix CI <ci@termix.dev>
Co-authored-by: Claude <noreply@anthropic.com>

* fix: file manager incorrectly decoding/encoding when editing files (made base64/utf8 dependent)

* fix: build error on docker

---------

Co-authored-by: Jefferson Nunn <89030989+jeffersonwarrior@users.noreply.github.com>
Co-authored-by: Termix CI <ci@termix.dev>
Co-authored-by: Claude <noreply@anthropic.com>

* Increase max old space size for npm builds

* Increase Node.js memory limit in Dockerfile

* Remove NODE_OPTIONS from build commands in Dockerfile

* Change runner to blacksmith-4vcpu-ubuntu-2404

* fix: build error on docker

* fix: prevent owner permission loss when sharing host to own role

Fixes #391

---------

Co-authored-by: Luke Gustafson <88517757+LukeGus@users.noreply.github.com>
Co-authored-by: Gaylord Julien <g.j@mailbox.org>
Co-authored-by: Jefferson Nunn <89030989+jeffersonwarrior@users.noreply.github.com>
Co-authored-by: Termix CI <ci@termix.dev>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: LukeGus <bugattiguy527@gmail.com>

* fix: SSH key passphrase not passed for Docker and Tunnel (#521)

* perf: optimize Host Manager for large host lists

- Add ServerStatusContext for shared status polling (reduces API calls from N to 1)
- Move TooltipProvider to component root (eliminates N context instances)
- Add pagination with "Show More" button (limits initial DOM nodes per folder)

Fixes performance issues when managing ~1000 hosts with status monitoring enabled.

* fix: SSH key passphrase not passed to ssh2 for Docker and Tunnel

Database field is `key_password` but code used `keyPassword`.
Added fallback to check both field names.

Affected:
- docker.ts: Docker SSH connections with encrypted keys
- tunnel.ts: Tunnel connections with encrypted keys

---------

Co-authored-by: Luke Gustafson <88517757+LukeGus@users.noreply.github.com>

* perf: optimize Host Manager for large host lists (#520)

- Add ServerStatusContext for shared status polling (reduces API calls from N to 1)
- Move TooltipProvider to component root (eliminates N context instances)
- Add pagination with "Show More" button (limits initial DOM nodes per folder)

Fixes performance issues when managing ~1000 hosts with status monitoring enabled.

Co-authored-by: Luke Gustafson <88517757+LukeGus@users.noreply.github.com>

* fix: add missing mimeTypes definition for image preview (#518)

Fixes Termix-SSH/Support#408

* fix: prevent session reset when updating host properties (#517)

Move WebSocket cleanup logic to a separate unmount-only effect to
prevent SSH sessions from being closed when host properties are updated.

Closes Termix-SSH/Support#401

* fix: backend type error

* feat: make terminal connections more resilient, added connection log, and fixed https/proxy reconnection loop (issue #385)

* feat: improved conneciton log ui/logic

* feat: improved conneciton log ui/logic

* feat: expanded connection log to work across all components (readying for release)

* feat: update readme

* feat: update readme

* fix: build error

* fix: build error

* fix: changed ver

* chore: clean up

* chore: continue clean up

* fix: remove attempts remaining and fix electron errors and some connection log ui inconsistencies

* fix: added missing nginx routes and fixed sudo password copy with sudo password autofil field

* fix: update readme and run cleaner

* fix: update readme

* fix: update readme

* fix: update readme

* feat: update chinese readme

---------

Co-authored-by: Steven Josefs <s.josefs@gmx.de>
Co-authored-by: Jefferson Nunn <89030989+jeffersonwarrior@users.noreply.github.com>
Co-authored-by: Termix CI <ci@termix.dev>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Nunzio Marfè <nunzio.marfe@protonmail.com>
Co-authored-by: Gaylord Julien <g.j@mailbox.org>
Co-authored-by: ZacharyZcR <zacharyzcr1984@gmail.com>
Co-authored-by: Aditya Tawade <36890395+aditya-tawade@users.noreply.github.com>
This commit is contained in:
Luke Gustafson
2026-01-24 19:49:42 -06:00
committed by GitHub
parent 366667ca92
commit 536dc35258
169 changed files with 79905 additions and 47362 deletions
+382
View File
@@ -0,0 +1,382 @@
import { Client } from "ssh2";
import type { WebSocket } from "ws";
import { sshLogger } from "../utils/logger.js";
import { getDb } from "../database/db/index.js";
import { sshCredentials, sshData } from "../database/db/schema.js";
import { eq, and } from "drizzle-orm";
import { SimpleDBOps } from "../utils/simple-db-ops.js";
import { UserCrypto } from "../utils/user-crypto.js";
interface ResolvedCredentials {
username: string;
password?: string;
key?: Buffer;
keyPassword?: string;
authType?: string;
}
interface HostConfig {
id: number;
ip: string;
port: number;
username: string;
password?: string;
key?: string;
keyPassword?: string;
keyType?: string;
authType?: string;
credentialId?: number;
userId?: string;
forceKeyboardInteractive?: boolean;
}
interface AuthContext {
userId: string;
ws: WebSocket;
hostId: number;
isKeyboardInteractive: boolean;
keyboardInteractiveResponded: boolean;
keyboardInteractiveFinish: ((responses: string[]) => void) | null;
totpPromptSent: boolean;
warpgateAuthPromptSent: boolean;
totpTimeout: NodeJS.Timeout | null;
warpgateAuthTimeout: NodeJS.Timeout | null;
totpAttempts: number;
}
const userCrypto = UserCrypto.getInstance();
const MAX_TOTP_ATTEMPTS = 3;
export class SSHAuthManager {
public context: AuthContext;
constructor(context: AuthContext) {
this.context = context;
}
async resolveCredentials(
hostConfig: HostConfig,
): Promise<ResolvedCredentials> {
let resolvedCredentials: ResolvedCredentials = {
username: hostConfig.username,
authType: hostConfig.authType || "none",
};
if (hostConfig.credentialId) {
const credentials = await SimpleDBOps.select(
getDb()
.select()
.from(sshCredentials)
.where(
and(
eq(sshCredentials.id, hostConfig.credentialId),
eq(sshCredentials.userId, this.context.userId),
),
),
"ssh_credentials",
this.context.userId,
);
if (credentials.length > 0) {
const cred = credentials[0];
resolvedCredentials = {
username: (cred.username as string) || hostConfig.username,
password: (cred.password as string) || undefined,
key: cred.privateKey
? Buffer.from(cred.privateKey as string)
: undefined,
keyPassword: (cred.passphrase as string) || undefined,
authType: (cred.authType as string) || "none",
};
}
} else {
if (hostConfig.password) {
resolvedCredentials.password = hostConfig.password;
resolvedCredentials.authType = "password";
}
if (hostConfig.key) {
resolvedCredentials.key = Buffer.from(hostConfig.key, "utf8");
resolvedCredentials.authType = "key";
if (hostConfig.keyPassword) {
resolvedCredentials.keyPassword = hostConfig.keyPassword;
}
}
}
return resolvedCredentials;
}
handleKeyboardInteractive(
name: string,
instructions: string,
instructionsLang: string,
prompts: Array<{ prompt: string; echo: boolean }>,
finish: (responses: string[]) => void,
resolvedCredentials: ResolvedCredentials,
): void {
this.context.isKeyboardInteractive = true;
const promptTexts = prompts.map((p) => p.prompt);
const warpgatePattern = /warpgate\s+authentication/i;
const isWarpgate =
warpgatePattern.test(name) ||
warpgatePattern.test(instructions) ||
promptTexts.some((p) => warpgatePattern.test(p));
if (isWarpgate) {
this.handleWarpgateAuth(name, instructions, promptTexts, finish);
return;
}
const totpPromptIndex = prompts.findIndex((p) =>
/verification code|verification_code|token|otp|2fa|authenticator|google.*auth/i.test(
p.prompt,
),
);
if (totpPromptIndex !== -1) {
this.handleTotpAuth(
prompts,
totpPromptIndex,
finish,
resolvedCredentials,
);
return;
}
this.handlePasswordAuth(prompts, finish, resolvedCredentials);
}
private handleWarpgateAuth(
name: string,
instructions: string,
promptTexts: string[],
finish: (responses: string[]) => void,
): void {
const fullText = `${name}\n${instructions}\n${promptTexts.join("\n")}`;
const urlMatch = fullText.match(/https?:\/\/[^\s\n]+/i);
const keyMatch = fullText.match(
/security key[:\s]+([a-z0-9](?:\s+[a-z0-9]){3}|[a-z0-9]{4})/i,
);
if (urlMatch) {
this.context.keyboardInteractiveFinish = (responses: string[]) => {
finish([""]);
};
this.context.warpgateAuthPromptSent = true;
this.sendLog("auth", "info", "Warpgate authentication required");
this.context.ws.send(
JSON.stringify({
type: "warpgate_auth_required",
url: urlMatch[0],
securityKey: keyMatch ? keyMatch[1] : "N/A",
instructions: instructions,
}),
);
this.context.warpgateAuthTimeout = setTimeout(() => {
if (this.context.keyboardInteractiveFinish) {
this.context.keyboardInteractiveFinish = null;
this.context.warpgateAuthPromptSent = false;
sshLogger.warn("Warpgate authentication timeout", {
operation: "warpgate_timeout",
hostId: this.context.hostId,
});
this.context.ws.send(
JSON.stringify({
type: "error",
message: "Warpgate authentication timeout. Please reconnect.",
}),
);
}
}, 300000);
}
}
private handleTotpAuth(
prompts: Array<{ prompt: string; echo: boolean }>,
totpPromptIndex: number,
finish: (responses: string[]) => void,
resolvedCredentials: ResolvedCredentials,
): void {
if (this.context.totpPromptSent) {
sshLogger.warn("TOTP prompt asked again - invalid code", {
operation: "ssh_keyboard_interactive_totp_retry",
hostId: this.context.hostId,
});
this.sendLog("auth", "warning", "Invalid TOTP code");
this.context.ws.send(
JSON.stringify({
type: "totp_retry",
}),
);
return;
}
this.context.totpPromptSent = true;
this.context.keyboardInteractiveResponded = true;
this.context.keyboardInteractiveFinish = (totpResponses: string[]) => {
const totpCode = (totpResponses[0] || "").trim();
const responses = prompts.map((p, index) => {
if (index === totpPromptIndex) {
return totpCode;
}
if (/password/i.test(p.prompt) && resolvedCredentials.password) {
return resolvedCredentials.password;
}
return "";
});
finish(responses);
};
if (this.context.totpTimeout) {
clearTimeout(this.context.totpTimeout);
}
this.context.totpTimeout = setTimeout(() => {
if (this.context.keyboardInteractiveFinish) {
this.context.keyboardInteractiveFinish = null;
this.context.totpPromptSent = false;
sshLogger.warn("TOTP prompt timeout", {
operation: "totp_timeout",
hostId: this.context.hostId,
});
this.context.ws.send(
JSON.stringify({
type: "error",
message: "TOTP verification timeout. Please reconnect.",
}),
);
}
}, 180000);
this.sendLog("auth", "info", "TOTP verification required");
this.context.ws.send(
JSON.stringify({
type: "totp_required",
prompt: prompts[totpPromptIndex].prompt,
}),
);
}
private handlePasswordAuth(
prompts: Array<{ prompt: string; echo: boolean }>,
finish: (responses: string[]) => void,
resolvedCredentials: ResolvedCredentials,
): void {
const hasStoredPassword =
resolvedCredentials.password && resolvedCredentials.authType !== "none";
const passwordPromptIndex = prompts.findIndex((p) =>
/password/i.test(p.prompt),
);
if (!hasStoredPassword && passwordPromptIndex !== -1) {
if (this.context.keyboardInteractiveResponded) {
return;
}
this.context.keyboardInteractiveResponded = true;
this.context.keyboardInteractiveFinish = (userResponses: string[]) => {
const userInput = (userResponses[0] || "").trim();
const responses = prompts.map((p, index) => {
if (index === passwordPromptIndex) {
return userInput;
}
return "";
});
finish(responses);
};
if (this.context.totpTimeout) {
clearTimeout(this.context.totpTimeout);
}
this.context.totpTimeout = setTimeout(() => {
if (this.context.keyboardInteractiveFinish) {
this.context.keyboardInteractiveFinish = null;
this.context.keyboardInteractiveResponded = false;
sshLogger.warn("Password prompt timeout", {
operation: "password_timeout",
hostId: this.context.hostId,
});
this.context.ws.send(
JSON.stringify({
type: "error",
message: "Password verification timeout. Please reconnect.",
}),
);
}
}, 180000);
this.sendLog("auth", "info", "Password authentication required");
this.context.ws.send(
JSON.stringify({
type: "password_required",
prompt: prompts[passwordPromptIndex].prompt,
}),
);
return;
}
const responses = prompts.map((p) => {
if (/password/i.test(p.prompt) && resolvedCredentials.password) {
return resolvedCredentials.password;
}
return "";
});
finish(responses);
}
sendLog(
stage: string,
level: string,
message: string,
details?: Record<string, any>,
): void {
this.context.ws.send(
JSON.stringify({
type: "connection_log",
data: {
stage,
level,
message,
details,
},
}),
);
}
cleanup(): void {
if (this.context.totpTimeout) {
clearTimeout(this.context.totpTimeout);
this.context.totpTimeout = null;
}
if (this.context.warpgateAuthTimeout) {
clearTimeout(this.context.warpgateAuthTimeout);
this.context.warpgateAuthTimeout = null;
}
this.context.keyboardInteractiveFinish = null;
this.context.totpPromptSent = false;
this.context.warpgateAuthPromptSent = false;
this.context.keyboardInteractiveResponded = false;
}
}
+995 -22
View File
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+547 -10
View File
@@ -11,6 +11,7 @@ import { SimpleDBOps } from "../utils/simple-db-ops.js";
import { AuthManager } from "../utils/auth-manager.js";
import { PermissionManager } from "../utils/permission-manager.js";
import type { AuthenticatedRequest, ProxyNode } from "../../types/index.js";
import type { LogEntry, ConnectionStage } from "../../types/connection-log.js";
import { collectCpuMetrics } from "./widgets/cpu-collector.js";
import { collectMemoryMetrics } from "./widgets/memory-collector.js";
import { collectDiskMetrics } from "./widgets/disk-collector.js";
@@ -19,8 +20,24 @@ import { collectUptimeMetrics } from "./widgets/uptime-collector.js";
import { collectProcessesMetrics } from "./widgets/processes-collector.js";
import { collectSystemMetrics } from "./widgets/system-collector.js";
import { collectLoginStats } from "./widgets/login-stats-collector.js";
import { collectPortsMetrics } from "./widgets/ports-collector.js";
import { collectFirewallMetrics } from "./widgets/firewall-collector.js";
import { createSocks5Connection } from "../utils/socks5-helper.js";
function createConnectionLog(
type: "info" | "success" | "warning" | "error",
stage: ConnectionStage,
message: string,
details?: Record<string, any>,
): Omit<LogEntry, "id" | "timestamp"> {
return {
type,
stage,
message,
details,
};
}
async function resolveJumpHost(
hostId: number,
userId: string,
@@ -1283,12 +1300,6 @@ class PollingManager {
for (const [sessionId, viewer] of this.viewerDetails.entries()) {
if (now - viewer.lastHeartbeat > maxInactivity) {
statsLogger.warn("Cleaning up inactive viewer", {
operation: "cleanup_inactive_viewer",
sessionId,
hostId: viewer.hostId,
inactiveFor: Math.floor((now - viewer.lastHeartbeat) / 1000),
});
this.unregisterViewer(viewer.hostId, sessionId);
}
}
@@ -1782,6 +1793,62 @@ async function collectMetrics(host: SSHHostWithCredentials): Promise<{
login_stats = await collectLoginStats(client);
} catch (e) {}
let ports: {
source: "ss" | "netstat" | "none";
ports: Array<{
protocol: "tcp" | "udp";
localAddress: string;
localPort: number;
state?: string;
pid?: number;
process?: string;
}>;
} = {
source: "none",
ports: [],
};
try {
ports = await collectPortsMetrics(client);
} catch (e) {
statsLogger.debug("Failed to collect ports metrics", {
operation: "ports_metrics_failed",
error: e instanceof Error ? e.message : String(e),
});
}
let firewall: {
type: "iptables" | "nftables" | "none";
status: "active" | "inactive" | "unknown";
chains: Array<{
name: string;
policy: string;
rules: Array<{
chain: string;
target: string;
protocol: string;
source: string;
destination: string;
dport?: string;
sport?: string;
state?: string;
interface?: string;
extra?: string;
}>;
}>;
} = {
type: "none",
status: "unknown",
chains: [],
};
try {
firewall = await collectFirewallMetrics(client);
} catch (e) {
statsLogger.debug("Failed to collect firewall metrics", {
operation: "firewall_metrics_failed",
error: e instanceof Error ? e.message : String(e),
});
}
const result = {
cpu,
memory,
@@ -1791,6 +1858,8 @@ async function collectMetrics(host: SSHHostWithCredentials): Promise<{
processes,
system,
login_stats,
ports,
firewall,
};
metricsCache.set(host.id, result);
@@ -1864,6 +1933,20 @@ function tcpPing(
});
}
/**
* @openapi
* /status:
* get:
* summary: Get all host statuses
* description: Retrieves the status of all hosts for the authenticated user.
* tags:
* - Server Stats
* responses:
* 200:
* description: A map of host IDs to their status entries.
* 401:
* description: Session expired - please log in again.
*/
app.get("/status", async (req, res) => {
const userId = (req as AuthenticatedRequest).userId;
@@ -1886,6 +1969,28 @@ app.get("/status", async (req, res) => {
res.json(result);
});
/**
* @openapi
* /status/{id}:
* get:
* summary: Get host status by ID
* description: Retrieves the status of a specific host by its ID.
* tags:
* - Server Stats
* parameters:
* - in: path
* name: id
* required: true
* schema:
* type: integer
* responses:
* 200:
* description: Host status entry.
* 401:
* description: Session expired - please log in again.
* 404:
* description: Status not available.
*/
app.get("/status/:id", validateHostId, async (req, res) => {
const id = Number(req.params.id);
const userId = (req as AuthenticatedRequest).userId;
@@ -1910,6 +2015,20 @@ app.get("/status/:id", validateHostId, async (req, res) => {
res.json(statusEntry);
});
/**
* @openapi
* /clear-connections:
* post:
* summary: Clear all SSH connections
* description: Clears all SSH connections from the connection pool.
* tags:
* - Server Stats
* responses:
* 200:
* description: All SSH connections cleared.
* 401:
* description: Session expired - please log in again.
*/
app.post("/clear-connections", async (req, res) => {
const userId = (req as AuthenticatedRequest).userId;
@@ -1924,6 +2043,20 @@ app.post("/clear-connections", async (req, res) => {
res.json({ message: "All SSH connections cleared" });
});
/**
* @openapi
* /refresh:
* post:
* summary: Refresh polling
* description: Clears all SSH connections and refreshes host polling.
* tags:
* - Server Stats
* responses:
* 200:
* description: Polling refreshed.
* 401:
* description: Session expired - please log in again.
*/
app.post("/refresh", async (req, res) => {
const userId = (req as AuthenticatedRequest).userId;
@@ -1940,6 +2073,35 @@ app.post("/refresh", async (req, res) => {
res.json({ message: "Polling refreshed" });
});
/**
* @openapi
* /host-updated:
* post:
* summary: Start polling for updated host
* description: Starts polling for a specific host after it has been updated.
* tags:
* - Server Stats
* requestBody:
* required: true
* content:
* application/json:
* schema:
* type: object
* properties:
* hostId:
* type: integer
* responses:
* 200:
* description: Host polling started.
* 400:
* description: Invalid hostId.
* 401:
* description: Session expired - please log in again.
* 404:
* description: Host not found.
* 500:
* description: Failed to start polling.
*/
app.post("/host-updated", async (req, res) => {
const userId = (req as AuthenticatedRequest).userId;
const { hostId } = req.body;
@@ -1975,6 +2137,33 @@ app.post("/host-updated", async (req, res) => {
}
});
/**
* @openapi
* /host-deleted:
* post:
* summary: Stop polling for deleted host
* description: Stops polling for a specific host after it has been deleted.
* tags:
* - Server Stats
* requestBody:
* required: true
* content:
* application/json:
* schema:
* type: object
* properties:
* hostId:
* type: integer
* responses:
* 200:
* description: Host polling stopped.
* 400:
* description: Invalid hostId.
* 401:
* description: Session expired - please log in again.
* 500:
* description: Failed to stop polling.
*/
app.post("/host-deleted", async (req, res) => {
const userId = (req as AuthenticatedRequest).userId;
const { hostId } = req.body;
@@ -2003,6 +2192,28 @@ app.post("/host-deleted", async (req, res) => {
}
});
/**
* @openapi
* /metrics/{id}:
* get:
* summary: Get host metrics
* description: Retrieves current metrics for a specific host including CPU, memory, disk, network, processes, and system information.
* tags:
* - Server Stats
* parameters:
* - in: path
* name: id
* required: true
* schema:
* type: integer
* responses:
* 200:
* description: Host metrics data.
* 401:
* description: Session expired - please log in again.
* 404:
* description: Metrics not available.
*/
app.get("/metrics/:id", validateHostId, async (req, res) => {
const id = Number(req.params.id);
const userId = (req as AuthenticatedRequest).userId;
@@ -2040,28 +2251,102 @@ app.get("/metrics/:id", validateHostId, async (req, res) => {
});
});
/**
* @openapi
* /metrics/start/{id}:
* post:
* summary: Start metrics collection
* description: Establishes an SSH connection and starts collecting metrics for a specific host.
* tags:
* - Server Stats
* parameters:
* - in: path
* name: id
* required: true
* schema:
* type: integer
* responses:
* 200:
* description: Metrics collection started successfully, or TOTP required.
* 401:
* description: Session expired - please log in again.
* 404:
* description: Host not found.
* 500:
* description: Failed to start metrics collection.
*/
app.post("/metrics/start/:id", validateHostId, async (req, res) => {
const id = Number(req.params.id);
const userId = (req as AuthenticatedRequest).userId;
const connectionLogs: Array<Omit<LogEntry, "id" | "timestamp">> = [];
if (!SimpleDBOps.isUserDataUnlocked(userId)) {
connectionLogs.push(
createConnectionLog("error", "stats_connecting", "Session expired"),
);
return res.status(401).json({
error: "Session expired - please log in again",
code: "SESSION_EXPIRED",
connectionLogs,
});
}
try {
const host = await fetchHostById(id, userId);
if (!host) {
return res.status(404).json({ error: "Host not found" });
connectionLogs.push(
createConnectionLog("error", "stats_connecting", "Host not found"),
);
return res.status(404).json({ error: "Host not found", connectionLogs });
}
connectionLogs.push(
createConnectionLog(
"info",
"stats_connecting",
"Starting metrics collection",
),
);
connectionLogs.push(
createConnectionLog("info", "dns", `Resolving DNS for ${host.ip}`),
);
connectionLogs.push(
createConnectionLog(
"info",
"tcp",
`Connecting to ${host.ip}:${host.port}`,
),
);
connectionLogs.push(
createConnectionLog("info", "handshake", "Initiating SSH handshake"),
);
if (host.authType === "password") {
connectionLogs.push(
createConnectionLog("info", "auth", "Authenticating with password"),
);
} else if (host.authType === "key") {
connectionLogs.push(
createConnectionLog("info", "auth", "Authenticating with SSH key"),
);
}
const sessionKey = getSessionKey(host.id, userId);
const existingSession = metricsSessions[sessionKey];
if (existingSession && existingSession.isConnected) {
return res.json({ success: true });
connectionLogs.push(
createConnectionLog(
"success",
"stats_polling",
"Using existing metrics session",
),
);
return res.json({ success: true, connectionLogs });
}
const config = buildSshConfig(host);
@@ -2113,6 +2398,14 @@ app.post("/metrics/start/:id", validateHostId, async (req, res) => {
totpAttempts: 0,
};
connectionLogs.push(
createConnectionLog(
"info",
"stats_totp",
"TOTP verification required",
),
);
clearTimeout(timeout);
if (!isResolved) {
isResolved = true;
@@ -2141,6 +2434,22 @@ app.post("/metrics/start/:id", validateHostId, async (req, res) => {
if (!isResolved) {
isResolved = true;
connectionLogs.push(
createConnectionLog(
"success",
"connected",
"SSH connection established successfully",
),
);
connectionLogs.push(
createConnectionLog(
"success",
"stats_polling",
"Metrics session established",
),
);
metricsSessions[sessionKey] = {
client,
isConnected: true,
@@ -2162,10 +2471,73 @@ app.post("/metrics/start/:id", validateHostId, async (req, res) => {
clearTimeout(timeout);
if (!isResolved) {
isResolved = true;
const errorMessage =
error instanceof Error ? error.message : String(error);
let errorStage: ConnectionStage = "error";
if (
errorMessage.includes("ENOTFOUND") ||
errorMessage.includes("getaddrinfo")
) {
errorStage = "dns";
connectionLogs.push(
createConnectionLog(
"error",
errorStage,
`DNS resolution failed: ${errorMessage}`,
),
);
} else if (
errorMessage.includes("ECONNREFUSED") ||
errorMessage.includes("ETIMEDOUT")
) {
errorStage = "tcp";
connectionLogs.push(
createConnectionLog(
"error",
errorStage,
`TCP connection failed: ${errorMessage}`,
),
);
} else if (
errorMessage.includes("handshake") ||
errorMessage.includes("key exchange")
) {
errorStage = "handshake";
connectionLogs.push(
createConnectionLog(
"error",
errorStage,
`SSH handshake failed: ${errorMessage}`,
),
);
} else if (
errorMessage.includes("authentication") ||
errorMessage.includes("Authentication")
) {
errorStage = "auth";
connectionLogs.push(
createConnectionLog(
"error",
errorStage,
`Authentication failed: ${errorMessage}`,
),
);
} else {
connectionLogs.push(
createConnectionLog(
"error",
"error",
`SSH connection failed: ${errorMessage}`,
),
);
}
statsLogger.error("SSH connection error in metrics/start", {
operation: "metrics_start_ssh_error",
hostId: host.id,
error: error instanceof Error ? error.message : String(error),
error: errorMessage,
});
reject(error);
}
@@ -2176,6 +2548,9 @@ app.post("/metrics/start/:id", validateHostId, async (req, res) => {
(host.socks5Host ||
(host.socks5ProxyChain && host.socks5ProxyChain.length > 0))
) {
connectionLogs.push(
createConnectionLog("info", "proxy", "Connecting via SOCKS5 proxy"),
);
createSocks5Connection(host.ip, host.port, {
useSocks5: host.useSocks5,
socks5Host: host.socks5Host,
@@ -2194,6 +2569,13 @@ app.post("/metrics/start/:id", validateHostId, async (req, res) => {
if (!isResolved) {
isResolved = true;
clearTimeout(timeout);
connectionLogs.push(
createConnectionLog(
"error",
"proxy",
`SOCKS5 proxy connection failed: ${error instanceof Error ? error.message : "Unknown error"}`,
),
);
reject(error);
}
});
@@ -2203,22 +2585,61 @@ app.post("/metrics/start/:id", validateHostId, async (req, res) => {
});
const result = await connectionPromise;
res.json(result);
res.json({ ...result, connectionLogs });
} catch (error) {
statsLogger.error("Failed to start metrics collection", {
operation: "metrics_start_error",
hostId: id,
error: error instanceof Error ? error.message : String(error),
});
connectionLogs.push(
createConnectionLog(
"error",
"stats_connecting",
`Failed to start metrics: ${error instanceof Error ? error.message : "Unknown error"}`,
),
);
res.status(500).json({
error:
error instanceof Error
? error.message
: "Failed to start metrics collection",
connectionLogs,
});
}
});
/**
* @openapi
* /metrics/stop/{id}:
* post:
* summary: Stop metrics collection
* description: Stops metrics collection for a specific host and cleans up the SSH session.
* tags:
* - Server Stats
* parameters:
* - in: path
* name: id
* required: true
* schema:
* type: integer
* requestBody:
* required: false
* content:
* application/json:
* schema:
* type: object
* properties:
* viewerSessionId:
* type: string
* responses:
* 200:
* description: Metrics collection stopped successfully.
* 401:
* description: Session expired - please log in again.
* 500:
* description: Failed to stop metrics collection.
*/
app.post("/metrics/stop/:id", validateHostId, async (req, res) => {
const id = Number(req.params.id);
const userId = (req as AuthenticatedRequest).userId;
@@ -2261,6 +2682,37 @@ app.post("/metrics/stop/:id", validateHostId, async (req, res) => {
}
});
/**
* @openapi
* /metrics/connect-totp:
* post:
* summary: Complete TOTP verification for metrics
* description: Verifies the TOTP code and completes the metrics SSH connection.
* tags:
* - Server Stats
* requestBody:
* required: true
* content:
* application/json:
* schema:
* type: object
* properties:
* sessionId:
* type: string
* totpCode:
* type: string
* responses:
* 200:
* description: TOTP verified, metrics connection established.
* 400:
* description: Missing sessionId or totpCode.
* 401:
* description: Session expired or invalid TOTP code.
* 404:
* description: TOTP session not found or expired.
* 500:
* description: Failed to verify TOTP.
*/
app.post("/metrics/connect-totp", async (req, res) => {
const { sessionId, totpCode } = req.body;
const userId = (req as AuthenticatedRequest).userId;
@@ -2396,6 +2848,35 @@ app.post("/metrics/connect-totp", async (req, res) => {
}
});
/**
* @openapi
* /metrics/heartbeat:
* post:
* summary: Update viewer heartbeat
* description: Updates the heartbeat timestamp for a metrics viewer session to keep it alive.
* tags:
* - Server Stats
* requestBody:
* required: true
* content:
* application/json:
* schema:
* type: object
* properties:
* viewerSessionId:
* type: string
* responses:
* 200:
* description: Heartbeat updated successfully.
* 400:
* description: Invalid viewerSessionId.
* 401:
* description: Session expired - please log in again.
* 404:
* description: Viewer session not found.
* 500:
* description: Failed to update heartbeat.
*/
app.post("/metrics/heartbeat", async (req, res) => {
const { viewerSessionId } = req.body;
const userId = (req as AuthenticatedRequest).userId;
@@ -2428,6 +2909,33 @@ app.post("/metrics/heartbeat", async (req, res) => {
}
});
/**
* @openapi
* /metrics/register-viewer:
* post:
* summary: Register metrics viewer
* description: Registers a new viewer session for a host to track who is viewing metrics.
* tags:
* - Server Stats
* requestBody:
* required: true
* content:
* application/json:
* schema:
* type: object
* properties:
* hostId:
* type: integer
* responses:
* 200:
* description: Viewer registered successfully.
* 400:
* description: Invalid hostId.
* 401:
* description: Session expired - please log in again.
* 500:
* description: Failed to register viewer.
*/
app.post("/metrics/register-viewer", async (req, res) => {
const { hostId } = req.body;
const userId = (req as AuthenticatedRequest).userId;
@@ -2458,6 +2966,35 @@ app.post("/metrics/register-viewer", async (req, res) => {
}
});
/**
* @openapi
* /metrics/unregister-viewer:
* post:
* summary: Unregister metrics viewer
* description: Unregisters a viewer session when they stop viewing metrics for a host.
* tags:
* - Server Stats
* requestBody:
* required: true
* content:
* application/json:
* schema:
* type: object
* properties:
* hostId:
* type: integer
* viewerSessionId:
* type: string
* responses:
* 200:
* description: Viewer unregistered successfully.
* 400:
* description: Invalid hostId or viewerSessionId.
* 401:
* description: Session expired - please log in again.
* 500:
* description: Failed to unregister viewer.
*/
app.post("/metrics/unregister-viewer", async (req, res) => {
const { hostId, viewerSessionId } = req.body;
const userId = (req as AuthenticatedRequest).userId;
+255 -143
View File
@@ -15,6 +15,7 @@ import { SimpleDBOps } from "../utils/simple-db-ops.js";
import { AuthManager } from "../utils/auth-manager.js";
import { UserCrypto } from "../utils/user-crypto.js";
import { createSocks5Connection } from "../utils/socks5-helper.js";
import { SSHAuthManager } from "./auth-manager.js";
interface ConnectToHostData {
cols: number;
@@ -331,7 +332,6 @@ wss.on("connection", async (ws: WebSocket, req) => {
let sshStream: ClientChannel | null = null;
let keyboardInteractiveFinish: ((responses: string[]) => void) | null = null;
let totpPromptSent = false;
let totpAttempts = 0;
let totpTimeout: NodeJS.Timeout | null = null;
let isKeyboardInteractive = false;
let keyboardInteractiveResponded = false;
@@ -339,6 +339,9 @@ wss.on("connection", async (ws: WebSocket, req) => {
let isConnected = false;
let isCleaningUp = false;
let isShellInitializing = false;
let warpgateAuthPromptSent = false;
let warpgateAuthTimeout: NodeJS.Timeout | null = null;
let isAwaitingAuthCredentials = false;
ws.on("close", () => {
const userWs = userConnections.get(userId);
@@ -454,7 +457,6 @@ wss.on("connection", async (ws: WebSocket, req) => {
totpTimeout = null;
}
const totpCode = totpData.code;
totpAttempts++;
keyboardInteractiveFinish([totpCode]);
keyboardInteractiveFinish = null;
totpPromptSent = false;
@@ -505,6 +507,19 @@ wss.on("connection", async (ws: WebSocket, req) => {
break;
}
case "warpgate_auth_continue": {
if (keyboardInteractiveFinish) {
if (warpgateAuthTimeout) {
clearTimeout(warpgateAuthTimeout);
warpgateAuthTimeout = null;
}
keyboardInteractiveFinish([""]);
keyboardInteractiveFinish = null;
warpgateAuthPromptSent = false;
}
break;
}
case "reconnect_with_credentials": {
const credentialsData = data as {
cols: number;
@@ -525,6 +540,7 @@ wss.on("connection", async (ws: WebSocket, req) => {
credentialsData.hostConfig.authType = "key";
}
isAwaitingAuthCredentials = false;
cleanupSSH();
const reconnectData: ConnectToHostData = {
@@ -576,6 +592,20 @@ wss.on("connection", async (ws: WebSocket, req) => {
credentialId,
} = hostConfig;
const sendLog = (
stage: string,
level: string,
message: string,
details?: Record<string, any>,
) => {
ws.send(
JSON.stringify({
type: "connection_log",
data: { stage, level, message, details },
}),
);
};
if (!username || typeof username !== "string" || username.trim() === "") {
sshLogger.error("Invalid username provided", undefined, {
operation: "ssh_connect",
@@ -634,6 +664,9 @@ wss.on("connection", async (ws: WebSocket, req) => {
isConnecting = true;
sshConn = new Client();
sendLog("dns", "info", `Starting address resolution of ${ip}`);
sendLog("tcp", "info", `Connecting to ${ip} port ${port}`);
const connectionTimeout = setTimeout(() => {
if (sshConn && isConnecting && !isConnected) {
sshLogger.error("SSH connection timeout", undefined, {
@@ -648,7 +681,7 @@ wss.on("connection", async (ws: WebSocket, req) => {
);
cleanupSSH(connectionTimeout);
}
}, 30000);
}, 120000);
let resolvedCredentials = { password, key, keyPassword, keyType, authType };
let authMethodNotAvailable = false;
@@ -713,6 +746,10 @@ wss.on("connection", async (ws: WebSocket, req) => {
sshConn.on("ready", () => {
clearTimeout(connectionTimeout);
sendLog("handshake", "success", "SSH handshake completed");
sendLog("auth", "success", `Authentication successful for ${username}`);
sendLog("connected", "success", "Connection established");
const conn = sshConn;
if (!conn || isCleaningUp || !sshConn) {
@@ -761,6 +798,36 @@ wss.on("connection", async (ws: WebSocket, req) => {
return;
}
sshLogger.info("Creating shell", {
operation: "ssh_shell_start",
hostId: id,
ip,
port,
username,
});
let shellCallbackReceived = false;
const shellTimeout = setTimeout(() => {
if (!shellCallbackReceived && isShellInitializing) {
sshLogger.error("Shell creation timeout - no response from server", {
operation: "ssh_shell_timeout",
hostId: id,
ip,
port,
username,
});
isShellInitializing = false;
ws.send(
JSON.stringify({
type: "error",
message:
"Shell creation timeout. The server may not support interactive shells or the connection was interrupted.",
}),
);
cleanupSSH(connectionTimeout);
}
}, 15000);
conn.shell(
{
rows: data.rows,
@@ -768,6 +835,8 @@ wss.on("connection", async (ws: WebSocket, req) => {
term: "xterm-256color",
} as PseudoTtyOptions,
(err, stream) => {
shellCallbackReceived = true;
clearTimeout(shellTimeout);
isShellInitializing = false;
if (err) {
@@ -784,6 +853,7 @@ wss.on("connection", async (ws: WebSocket, req) => {
message: "Shell error: " + err.message,
}),
);
cleanupSSH(connectionTimeout);
return;
}
@@ -902,21 +972,7 @@ wss.on("connection", async (ws: WebSocket, req) => {
sshConn.on("error", (err: Error) => {
clearTimeout(connectionTimeout);
if (
(authMethodNotAvailable && resolvedCredentials.authType === "none") ||
(resolvedCredentials.authType === "none" &&
err.message.includes("All configured authentication methods failed"))
) {
ws.send(
JSON.stringify({
type: "auth_method_not_available",
message:
"The server does not support keyboard-interactive authentication. Please provide credentials.",
}),
);
cleanupSSH(connectionTimeout);
return;
}
sendLog("error", "error", `Connection error: ${err.message}`);
sshLogger.error("SSH connection error", err, {
operation: "ssh_connect",
@@ -925,8 +981,99 @@ wss.on("connection", async (ws: WebSocket, req) => {
port,
username,
authType: resolvedCredentials.authType,
warpgateAuthPromptSent,
isKeyboardInteractive,
hasKeyboardInteractiveFinish: !!keyboardInteractiveFinish,
keyboardInteractiveResponded,
});
if (
authMethodNotAvailable &&
resolvedCredentials.authType === "none" &&
!isKeyboardInteractive
) {
sendLog(
"auth",
"error",
"Server does not support keyboard-interactive authentication",
);
clearTimeout(connectionTimeout);
isAwaitingAuthCredentials = true;
if (sshConn) {
try {
sshConn.end();
} catch (e) {}
sshConn = null;
}
isConnecting = false;
isConnected = false;
ws.send(
JSON.stringify({
type: "auth_method_not_available",
message:
"The server does not support keyboard-interactive authentication. Please provide credentials.",
}),
);
return;
}
if (
resolvedCredentials.authType === "none" &&
err.message.includes("All configured authentication methods failed") &&
!isKeyboardInteractive &&
!keyboardInteractiveResponded
) {
clearTimeout(connectionTimeout);
isAwaitingAuthCredentials = true;
if (sshConn) {
try {
sshConn.end();
} catch (e) {}
sshConn = null;
}
isConnecting = false;
isConnected = false;
ws.send(
JSON.stringify({
type: "auth_method_not_available",
message:
"The server does not support keyboard-interactive authentication. Please provide credentials.",
}),
);
return;
}
if (
isKeyboardInteractive &&
keyboardInteractiveFinish &&
err.message.includes("All configured authentication methods failed")
) {
sshLogger.warn(
"Authentication error during keyboard-interactive - SKIPPING cleanup, waiting for user response",
{
operation: "ssh_error_during_keyboard_interactive_skip_cleanup",
hostId: id,
error: err.message,
},
);
return;
}
sshLogger.error("Proceeding with cleanup after error", {
operation: "ssh_error_cleanup",
hostId: id,
error: err.message,
});
if (
err.message.includes("authentication") ||
err.message.includes("Authentication")
) {
sendLog("auth", "error", `Authentication failed: ${err.message}`);
} else {
sendLog("error", "error", `Connection failed: ${err.message}`);
}
let errorMessage = "SSH error: " + err.message;
if (err.message.includes("No matching key exchange algorithm")) {
errorMessage =
@@ -969,9 +1116,54 @@ wss.on("connection", async (ws: WebSocket, req) => {
sshConn.on("close", () => {
clearTimeout(connectionTimeout);
if (isAwaitingAuthCredentials) {
cleanupSSH(connectionTimeout);
return;
}
if (isShellInitializing || (isConnected && !sshStream)) {
sshLogger.warn("SSH connection closed during shell initialization", {
operation: "ssh_close_during_init",
hostId: id,
ip,
port,
username,
isShellInitializing,
hasStream: !!sshStream,
});
ws.send(
JSON.stringify({
type: "error",
message:
"Connection closed during shell initialization. The server may have rejected the shell request.",
}),
);
} else if (!sshStream) {
ws.send(
JSON.stringify({
type: "disconnected",
message: "Connection closed",
}),
);
}
cleanupSSH(connectionTimeout);
});
const sshAuthManager = new SSHAuthManager({
userId,
ws,
hostId: id || 0,
isKeyboardInteractive,
keyboardInteractiveResponded,
keyboardInteractiveFinish,
totpPromptSent,
warpgateAuthPromptSent,
totpTimeout,
warpgateAuthTimeout,
totpAttempts: 0,
});
sshConn.on(
"keyboard-interactive",
(
@@ -981,130 +1173,28 @@ wss.on("connection", async (ws: WebSocket, req) => {
prompts: Array<{ prompt: string; echo: boolean }>,
finish: (responses: string[]) => void,
) => {
isKeyboardInteractive = true;
const promptTexts = prompts.map((p) => p.prompt);
const totpPromptIndex = prompts.findIndex((p) =>
/verification code|verification_code|token|otp|2fa|authenticator|google.*auth/i.test(
p.prompt,
),
if (connectionTimeout) {
clearTimeout(connectionTimeout);
}
sshAuthManager.handleKeyboardInteractive(
name,
instructions,
instructionsLang,
prompts,
finish,
resolvedCredentials as any,
);
if (totpPromptIndex !== -1) {
if (totpPromptSent) {
sshLogger.warn("TOTP prompt asked again - ignoring duplicate", {
operation: "ssh_keyboard_interactive_totp_duplicate",
hostId: id,
prompts: promptTexts,
});
return;
}
totpPromptSent = true;
keyboardInteractiveResponded = true;
keyboardInteractiveFinish = (totpResponses: string[]) => {
const totpCode = (totpResponses[0] || "").trim();
const responses = prompts.map((p, index) => {
if (index === totpPromptIndex) {
return totpCode;
}
if (/password/i.test(p.prompt) && resolvedCredentials.password) {
return resolvedCredentials.password;
}
return "";
});
finish(responses);
};
totpTimeout = setTimeout(() => {
if (keyboardInteractiveFinish) {
keyboardInteractiveFinish = null;
totpPromptSent = false;
sshLogger.warn("TOTP prompt timeout", {
operation: "totp_timeout",
hostId: id,
});
ws.send(
JSON.stringify({
type: "error",
message: "TOTP verification timeout. Please reconnect.",
}),
);
cleanupSSH(connectionTimeout);
}
}, 180000);
ws.send(
JSON.stringify({
type: "totp_required",
prompt: prompts[totpPromptIndex].prompt,
}),
);
} else {
const hasStoredPassword =
resolvedCredentials.password &&
resolvedCredentials.authType !== "none";
const passwordPromptIndex = prompts.findIndex((p) =>
/password/i.test(p.prompt),
);
if (!hasStoredPassword && passwordPromptIndex !== -1) {
if (keyboardInteractiveResponded) {
return;
}
keyboardInteractiveResponded = true;
keyboardInteractiveFinish = (userResponses: string[]) => {
const userInput = (userResponses[0] || "").trim();
const responses = prompts.map((p, index) => {
if (index === passwordPromptIndex) {
return userInput;
}
return "";
});
finish(responses);
};
totpTimeout = setTimeout(() => {
if (keyboardInteractiveFinish) {
keyboardInteractiveFinish = null;
keyboardInteractiveResponded = false;
sshLogger.warn("Password prompt timeout", {
operation: "password_timeout",
hostId: id,
});
ws.send(
JSON.stringify({
type: "error",
message: "Password verification timeout. Please reconnect.",
}),
);
cleanupSSH(connectionTimeout);
}
}, 180000);
ws.send(
JSON.stringify({
type: "password_required",
prompt: prompts[passwordPromptIndex].prompt,
}),
);
return;
}
const responses = prompts.map((p) => {
if (/password/i.test(p.prompt) && resolvedCredentials.password) {
return resolvedCredentials.password;
}
return "";
});
finish(responses);
}
isKeyboardInteractive = sshAuthManager.context.isKeyboardInteractive;
keyboardInteractiveResponded =
sshAuthManager.context.keyboardInteractiveResponded;
keyboardInteractiveFinish =
sshAuthManager.context.keyboardInteractiveFinish;
totpPromptSent = sshAuthManager.context.totpPromptSent;
warpgateAuthPromptSent = sshAuthManager.context.warpgateAuthPromptSent;
totpTimeout = sshAuthManager.context.totpTimeout;
warpgateAuthTimeout = sshAuthManager.context.warpgateAuthTimeout;
},
);
@@ -1115,10 +1205,10 @@ wss.on("connection", async (ws: WebSocket, req) => {
tryKeyboard: true,
keepaliveInterval: 30000,
keepaliveCountMax: 3,
readyTimeout: 30000,
readyTimeout: 120000,
tcpKeepAlive: true,
tcpKeepAliveInitialDelay: 30000,
timeout: 30000,
timeout: 120000,
env: {
TERM: "xterm-256color",
LANG: "en_US.UTF-8",
@@ -1195,10 +1285,12 @@ wss.on("connection", async (ws: WebSocket, req) => {
}
connectConfig.password = resolvedCredentials.password;
sendLog("auth", "info", "Using password authentication");
} else if (
resolvedCredentials.authType === "key" &&
resolvedCredentials.key
) {
sendLog("auth", "info", "Using SSH key authentication");
try {
if (
!resolvedCredentials.key.includes("-----BEGIN") ||
@@ -1228,6 +1320,11 @@ wss.on("connection", async (ws: WebSocket, req) => {
return;
}
} else if (resolvedCredentials.authType === "key") {
sendLog(
"auth",
"error",
"SSH key authentication requested but no key provided",
);
sshLogger.error("SSH key authentication requested but no key provided");
ws.send(
JSON.stringify({
@@ -1237,6 +1334,7 @@ wss.on("connection", async (ws: WebSocket, req) => {
);
return;
} else {
sendLog("auth", "info", "Using keyboard-interactive authentication");
sshLogger.error("No valid authentication method provided");
ws.send(
JSON.stringify({
@@ -1333,6 +1431,12 @@ wss.on("connection", async (ws: WebSocket, req) => {
}
connectConfig.sock = stream;
sendLog(
"handshake",
"info",
"Starting SSH session through jump host",
);
sendLog("auth", "info", `Authenticating as ${username}`);
sshConn.connect(connectConfig);
});
} catch (error) {
@@ -1350,6 +1454,8 @@ wss.on("connection", async (ws: WebSocket, req) => {
return;
}
} else {
sendLog("handshake", "info", "Starting SSH session");
sendLog("auth", "info", `Authenticating as ${username}`);
sshConn.connect(connectConfig);
}
}
@@ -1391,6 +1497,11 @@ wss.on("connection", async (ws: WebSocket, req) => {
totpTimeout = null;
}
if (warpgateAuthTimeout) {
clearTimeout(warpgateAuthTimeout);
warpgateAuthTimeout = null;
}
if (sshStream) {
try {
sshStream.end();
@@ -1416,13 +1527,14 @@ wss.on("connection", async (ws: WebSocket, req) => {
}
totpPromptSent = false;
totpAttempts = 0;
warpgateAuthPromptSent = false;
isKeyboardInteractive = false;
keyboardInteractiveResponded = false;
keyboardInteractiveFinish = null;
isConnecting = false;
isConnected = false;
isCleaningUp = false;
isAwaitingAuthCredentials = false;
}
// Note: PTY-level keepalive (writing \x00 to the stream) was removed.
+242 -16
View File
@@ -548,9 +548,16 @@ async function connectSSHTunnel(
!tunnelConfig.sourceUsername ||
!tunnelConfig.sourceSSHPort
) {
tunnelLogger.error("Invalid tunnel connection details", {
operation: "tunnel_connect",
const missingFields = [];
if (!tunnelConfig) missingFields.push("tunnelConfig");
if (!tunnelConfig?.sourceIP) missingFields.push("sourceIP");
if (!tunnelConfig?.sourceUsername) missingFields.push("sourceUsername");
if (!tunnelConfig?.sourceSSHPort) missingFields.push("sourceSSHPort");
tunnelLogger.error("Invalid tunnel connection details", undefined, {
operation: "tunnel_connect_validation_failed",
tunnelName,
missingFields: missingFields.join(", "),
hasSourceIP: !!tunnelConfig?.sourceIP,
hasSourceUsername: !!tunnelConfig?.sourceUsername,
hasSourceSSHPort: !!tunnelConfig?.sourceSSHPort,
@@ -560,6 +567,7 @@ async function connectSSHTunnel(
status: CONNECTION_STATES.FAILED,
reason: "Missing required connection details",
});
tunnelConnecting.delete(tunnelName);
return;
}
@@ -600,12 +608,19 @@ async function connectSSHTunnel(
};
} else {
const errorMessage = `Cannot connect tunnel '${tunnelName}': shared credentials not available`;
tunnelLogger.error(errorMessage);
tunnelLogger.error(errorMessage, undefined, {
operation: "tunnel_shared_credentials_unavailable",
tunnelName,
requestingUserId: tunnelConfig.requestingUserId,
sourceUserId: tunnelConfig.sourceUserId,
sourceHostId: tunnelConfig.sourceHostId,
});
broadcastTunnelStatus(tunnelName, {
connected: false,
status: CONNECTION_STATES.FAILED,
reason: errorMessage,
});
tunnelConnecting.delete(tunnelName);
return;
}
}
@@ -662,12 +677,18 @@ async function connectSSHTunnel(
!resolvedEndpointCredentials.password
) {
const errorMessage = `Cannot connect tunnel '${tunnelName}': endpoint host requires password authentication but no plaintext password available. Enable autostart for endpoint host or configure credentials in tunnel connection.`;
tunnelLogger.error(errorMessage);
tunnelLogger.error(errorMessage, undefined, {
operation: "tunnel_endpoint_password_unavailable",
tunnelName,
endpointHost: `${tunnelConfig.endpointUsername}@${tunnelConfig.endpointIP}:${tunnelConfig.endpointPort}`,
endpointAuthMethod: resolvedEndpointCredentials.authMethod,
});
broadcastTunnelStatus(tunnelName, {
connected: false,
status: CONNECTION_STATES.FAILED,
reason: errorMessage,
});
tunnelConnecting.delete(tunnelName);
return;
}
@@ -676,12 +697,18 @@ async function connectSSHTunnel(
!resolvedEndpointCredentials.sshKey
) {
const errorMessage = `Cannot connect tunnel '${tunnelName}': endpoint host requires key authentication but no plaintext key available. Enable autostart for endpoint host or configure credentials in tunnel connection.`;
tunnelLogger.error(errorMessage);
tunnelLogger.error(errorMessage, undefined, {
operation: "tunnel_endpoint_key_unavailable",
tunnelName,
endpointHost: `${tunnelConfig.endpointUsername}@${tunnelConfig.endpointIP}:${tunnelConfig.endpointPort}`,
endpointAuthMethod: resolvedEndpointCredentials.authMethod,
});
broadcastTunnelStatus(tunnelName, {
connected: false,
status: CONNECTION_STATES.FAILED,
reason: errorMessage,
});
tunnelConnecting.delete(tunnelName);
return;
}
@@ -745,6 +772,19 @@ async function connectSSHTunnel(
return;
}
tunnelLogger.error(
`Tunnel connection timeout after 60 seconds for '${tunnelName}'`,
undefined,
{
operation: "tunnel_connection_timeout",
tunnelName,
sourceHost: `${tunnelConfig.sourceUsername}@${tunnelConfig.sourceIP}:${tunnelConfig.sourceSSHPort}`,
endpointHost: `${tunnelConfig.endpointUsername}@${tunnelConfig.endpointIP}:${tunnelConfig.endpointPort}`,
retryAttempt,
usingSocks5: tunnelConfig.useSocks5 || false,
},
);
try {
conn.end();
} catch (error) {}
@@ -763,7 +803,22 @@ async function connectSSHTunnel(
conn.on("error", (err) => {
clearTimeout(connectionTimeout);
tunnelLogger.error(`SSH error for '${tunnelName}': ${err.message}`);
const errorType = classifyError(err.message);
tunnelLogger.error(`Tunnel connection failed for '${tunnelName}'`, err, {
operation: "tunnel_connect_error",
tunnelName,
errorType,
errorMessage: err.message,
sourceHost: `${tunnelConfig.sourceUsername}@${tunnelConfig.sourceIP}:${tunnelConfig.sourceSSHPort}`,
endpointHost: `${tunnelConfig.endpointUsername}@${tunnelConfig.endpointIP}:${tunnelConfig.endpointPort}`,
tunnelType: tunnelConfig.tunnelType || "remote",
sourcePort: tunnelConfig.sourcePort,
retryAttempt,
usingSocks5: tunnelConfig.useSocks5 || false,
authMethod: tunnelConfig.sourceAuthMethod,
});
tunnelConnecting.delete(tunnelName);
@@ -771,8 +826,6 @@ async function connectSSHTunnel(
return;
}
const errorType = classifyError(err.message);
if (!manualDisconnects.has(tunnelName)) {
broadcastTunnelStatus(tunnelName, {
connected: false,
@@ -828,28 +881,49 @@ async function connectSSHTunnel(
return;
}
const tunnelType = tunnelConfig.tunnelType || "remote";
const tunnelFlag = tunnelType === "local" ? "-L" : "-R";
const portMapping =
tunnelType === "local"
? `${tunnelConfig.sourcePort}:${tunnelConfig.endpointIP}:${tunnelConfig.endpointPort}`
: `${tunnelConfig.endpointPort}:localhost:${tunnelConfig.sourcePort}`;
let tunnelCmd: string;
if (
resolvedEndpointCredentials.authMethod === "key" &&
resolvedEndpointCredentials.sshKey
) {
const keyFilePath = `/tmp/tunnel_key_${tunnelName.replace(/[^a-zA-Z0-9]/g, "_")}`;
tunnelCmd = `echo '${resolvedEndpointCredentials.sshKey}' > ${keyFilePath} && chmod 600 ${keyFilePath} && exec -a "${tunnelMarker}" ssh -i ${keyFilePath} -N -o StrictHostKeyChecking=no -o ExitOnForwardFailure=yes -o ServerAliveInterval=30 -o ServerAliveCountMax=3 -o GatewayPorts=yes -R ${tunnelConfig.endpointPort}:localhost:${tunnelConfig.sourcePort} ${tunnelConfig.endpointUsername}@${tunnelConfig.endpointIP} && rm -f ${keyFilePath}`;
tunnelCmd = `echo '${resolvedEndpointCredentials.sshKey}' > ${keyFilePath} && chmod 600 ${keyFilePath} && exec -a "${tunnelMarker}" ssh -i ${keyFilePath} -N -o StrictHostKeyChecking=no -o ExitOnForwardFailure=yes -o ServerAliveInterval=30 -o ServerAliveCountMax=3 -o GatewayPorts=yes ${tunnelFlag} ${portMapping} ${tunnelConfig.endpointUsername}@${tunnelConfig.endpointIP} && rm -f ${keyFilePath}`;
} else {
tunnelCmd = `exec -a "${tunnelMarker}" sshpass -p '${resolvedEndpointCredentials.password || ""}' ssh -N -o StrictHostKeyChecking=no -o ExitOnForwardFailure=yes -o ServerAliveInterval=30 -o ServerAliveCountMax=3 -o GatewayPorts=yes -R ${tunnelConfig.endpointPort}:localhost:${tunnelConfig.sourcePort} ${tunnelConfig.endpointUsername}@${tunnelConfig.endpointIP}`;
tunnelCmd = `exec -a "${tunnelMarker}" sshpass -p '${resolvedEndpointCredentials.password || ""}' ssh -N -o StrictHostKeyChecking=no -o ExitOnForwardFailure=yes -o ServerAliveInterval=30 -o ServerAliveCountMax=3 -o GatewayPorts=yes ${tunnelFlag} ${portMapping} ${tunnelConfig.endpointUsername}@${tunnelConfig.endpointIP}`;
}
conn.exec(tunnelCmd, (err, stream) => {
if (err) {
const errorType = classifyError(err.message);
tunnelLogger.error(
`Connection error for '${tunnelName}': ${err.message}`,
`Failed to execute tunnel command for '${tunnelName}'`,
err,
{
operation: "tunnel_exec_error",
tunnelName,
errorType,
errorMessage: err.message,
sourceHost: `${tunnelConfig.sourceUsername}@${tunnelConfig.sourceIP}:${tunnelConfig.sourceSSHPort}`,
endpointHost: `${tunnelConfig.endpointUsername}@${tunnelConfig.endpointIP}:${tunnelConfig.endpointPort}`,
tunnelType: tunnelConfig.tunnelType || "remote",
sourcePort: tunnelConfig.sourcePort,
endpointPort: tunnelConfig.endpointPort,
retryAttempt,
},
);
conn.end();
activeTunnels.delete(tunnelName);
const errorType = classifyError(err.message);
const shouldNotRetry =
errorType === "AUTHENTICATION_FAILED" ||
errorType === "CONNECTION_FAILED";
@@ -1079,12 +1153,23 @@ async function connectSSHTunnel(
) {
tunnelLogger.error(
`Invalid SSH key format for tunnel '${tunnelName}'. Key should contain both BEGIN and END markers`,
undefined,
{
operation: "tunnel_invalid_ssh_key_format",
tunnelName,
sourceHost: `${tunnelConfig.sourceUsername}@${tunnelConfig.sourceIP}:${tunnelConfig.sourceSSHPort}`,
keyType: resolvedSourceCredentials.keyType,
hasBeginMarker:
resolvedSourceCredentials.sshKey.includes("-----BEGIN"),
hasEndMarker: resolvedSourceCredentials.sshKey.includes("-----END"),
},
);
broadcastTunnelStatus(tunnelName, {
connected: false,
status: CONNECTION_STATES.FAILED,
reason: "Invalid SSH key format",
});
tunnelConnecting.delete(tunnelName);
return;
}
@@ -1105,12 +1190,20 @@ async function connectSSHTunnel(
} else if (resolvedSourceCredentials.authMethod === "key") {
tunnelLogger.error(
`SSH key authentication requested but no key provided for tunnel '${tunnelName}'`,
undefined,
{
operation: "tunnel_ssh_key_missing",
tunnelName,
sourceHost: `${tunnelConfig.sourceUsername}@${tunnelConfig.sourceIP}:${tunnelConfig.sourceSSHPort}`,
authMethod: resolvedSourceCredentials.authMethod,
},
);
broadcastTunnelStatus(tunnelName, {
connected: false,
status: CONNECTION_STATES.FAILED,
reason: "SSH key authentication requested but no key provided",
});
tunnelConnecting.delete(tunnelName);
return;
} else {
connOptions.password = resolvedSourceCredentials.password;
@@ -1152,10 +1245,16 @@ async function connectSSHTunnel(
}
} catch (socks5Error) {
tunnelLogger.error("SOCKS5 connection failed for tunnel", socks5Error, {
operation: "socks5_connect",
operation: "tunnel_socks5_connection_failed",
tunnelName,
sourceHost: `${tunnelConfig.sourceIP}:${tunnelConfig.sourceSSHPort}`,
proxyHost: tunnelConfig.socks5Host,
proxyPort: tunnelConfig.socks5Port || 1080,
hasProxyAuth: !!(
tunnelConfig.socks5Username && tunnelConfig.socks5Password
),
errorMessage:
socks5Error instanceof Error ? socks5Error.message : "Unknown error",
});
broadcastTunnelStatus(tunnelName, {
connected: false,
@@ -1166,6 +1265,7 @@ async function connectSSHTunnel(
? socks5Error.message
: "Unknown error"),
});
tunnelConnecting.delete(tunnelName);
return;
}
}
@@ -1302,7 +1402,9 @@ async function killRemoteTunnelByMarker(
}
conn.on("ready", () => {
const checkCmd = `ps aux | grep -E '(${tunnelMarker}|ssh.*-R.*${tunnelConfig.endpointPort}:localhost:${tunnelConfig.sourcePort}.*${tunnelConfig.endpointUsername}@${tunnelConfig.endpointIP}|sshpass.*ssh.*-R.*${tunnelConfig.endpointPort})' | grep -v grep`;
const tunnelType = tunnelConfig.tunnelType || "remote";
const tunnelFlag = tunnelType === "local" ? "-L" : "-R";
const checkCmd = `ps aux | grep -E '(${tunnelMarker}|ssh.*${tunnelFlag}.*${tunnelConfig.endpointPort}:.*:${tunnelConfig.sourcePort}.*${tunnelConfig.endpointUsername}@${tunnelConfig.endpointIP}|sshpass.*ssh.*${tunnelFlag})' | grep -v grep`;
conn.exec(checkCmd, (_err, stream) => {
let foundProcesses = false;
@@ -1323,8 +1425,8 @@ async function killRemoteTunnelByMarker(
const killCmds = [
`pkill -TERM -f '${tunnelMarker}'`,
`sleep 1 && pkill -f 'ssh.*-R.*${tunnelConfig.endpointPort}:localhost:${tunnelConfig.sourcePort}.*${tunnelConfig.endpointUsername}@${tunnelConfig.endpointIP}'`,
`sleep 1 && pkill -f 'sshpass.*ssh.*-R.*${tunnelConfig.endpointPort}'`,
`sleep 1 && pkill -f 'ssh.*${tunnelFlag}.*${tunnelConfig.endpointPort}:.*:${tunnelConfig.sourcePort}.*${tunnelConfig.endpointUsername}@${tunnelConfig.endpointIP}'`,
`sleep 1 && pkill -f 'sshpass.*ssh.*${tunnelFlag}.*${tunnelConfig.endpointPort}'`,
`sleep 2 && pkill -9 -f '${tunnelMarker}'`,
];
@@ -1450,10 +1552,42 @@ async function killRemoteTunnelByMarker(
}
}
/**
* @openapi
* /ssh/tunnel/status:
* get:
* summary: Get all tunnel statuses
* description: Retrieves the status of all SSH tunnels.
* tags:
* - SSH Tunnels
* responses:
* 200:
* description: A list of all tunnel statuses.
*/
app.get("/ssh/tunnel/status", (req, res) => {
res.json(getAllTunnelStatus());
});
/**
* @openapi
* /ssh/tunnel/status/{tunnelName}:
* get:
* summary: Get tunnel status by name
* description: Retrieves the status of a specific SSH tunnel by its name.
* tags:
* - SSH Tunnels
* parameters:
* - in: path
* name: tunnelName
* required: true
* schema:
* type: string
* responses:
* 200:
* description: Tunnel status.
* 404:
* description: Tunnel not found.
*/
app.get("/ssh/tunnel/status/:tunnelName", (req, res) => {
const { tunnelName } = req.params;
const status = connectionStatus.get(tunnelName);
@@ -1465,6 +1599,39 @@ app.get("/ssh/tunnel/status/:tunnelName", (req, res) => {
res.json({ name: tunnelName, status });
});
/**
* @openapi
* /ssh/tunnel/connect:
* post:
* summary: Connect SSH tunnel
* description: Establishes an SSH tunnel connection with the specified configuration.
* tags:
* - SSH Tunnels
* requestBody:
* required: true
* content:
* application/json:
* schema:
* type: object
* properties:
* name:
* type: string
* sourceHostId:
* type: integer
* tunnelIndex:
* type: integer
* responses:
* 200:
* description: Connection request received.
* 400:
* description: Invalid tunnel configuration.
* 401:
* description: Authentication required.
* 403:
* description: Access denied to this host.
* 500:
* description: Failed to connect tunnel.
*/
app.post(
"/ssh/tunnel/connect",
authenticateJWT,
@@ -1619,6 +1786,35 @@ app.post(
},
);
/**
* @openapi
* /ssh/tunnel/disconnect:
* post:
* summary: Disconnect SSH tunnel
* description: Disconnects an active SSH tunnel.
* tags:
* - SSH Tunnels
* requestBody:
* required: true
* content:
* application/json:
* schema:
* type: object
* properties:
* tunnelName:
* type: string
* responses:
* 200:
* description: Disconnect request received.
* 400:
* description: Tunnel name required.
* 401:
* description: Authentication required.
* 403:
* description: Access denied.
* 500:
* description: Failed to disconnect tunnel.
*/
app.post(
"/ssh/tunnel/disconnect",
authenticateJWT,
@@ -1683,6 +1879,35 @@ app.post(
},
);
/**
* @openapi
* /ssh/tunnel/cancel:
* post:
* summary: Cancel tunnel retry
* description: Cancels the retry mechanism for a failed SSH tunnel connection.
* tags:
* - SSH Tunnels
* requestBody:
* required: true
* content:
* application/json:
* schema:
* type: object
* properties:
* tunnelName:
* type: string
* responses:
* 200:
* description: Cancel request received.
* 400:
* description: Tunnel name required.
* 401:
* description: Authentication required.
* 403:
* description: Access denied.
* 500:
* description: Failed to cancel tunnel retry.
*/
app.post(
"/ssh/tunnel/cancel",
authenticateJWT,
@@ -1806,6 +2031,7 @@ async function initializeAutoStartTunnels(): Promise<void> {
tunnelConnection.endpointHost,
tunnelConnection.endpointPort,
),
tunnelType: tunnelConnection.tunnelType || "remote",
sourceHostId: host.id,
tunnelIndex: tunnelIndex,
hostName: host.name || `${host.username}@${host.ip}`,
@@ -0,0 +1,254 @@
import type { Client } from "ssh2";
import { execCommand } from "./common-utils.js";
import type {
FirewallMetrics,
FirewallChain,
FirewallRule,
} from "../../../types/stats-widgets.js";
function parseIptablesRule(line: string): FirewallRule | null {
if (!line.startsWith("-A ")) return null;
const rule: FirewallRule = {
chain: "",
target: "",
protocol: "all",
source: "0.0.0.0/0",
destination: "0.0.0.0/0",
};
const chainMatch = line.match(/^-A\s+(\S+)/);
if (chainMatch) {
rule.chain = chainMatch[1];
}
const targetMatch = line.match(/-j\s+(\S+)/);
if (targetMatch) {
rule.target = targetMatch[1];
}
const protocolMatch = line.match(/-p\s+(\S+)/);
if (protocolMatch) {
rule.protocol = protocolMatch[1];
}
const sourceMatch = line.match(/-s\s+(\S+)/);
if (sourceMatch) {
rule.source = sourceMatch[1];
}
const destMatch = line.match(/-d\s+(\S+)/);
if (destMatch) {
rule.destination = destMatch[1];
}
const dportMatch = line.match(/--dport\s+(\S+)/);
if (dportMatch) {
rule.dport = dportMatch[1];
}
const sportMatch = line.match(/--sport\s+(\S+)/);
if (sportMatch) {
rule.sport = sportMatch[1];
}
const stateMatch = line.match(/--state\s+(\S+)/);
if (stateMatch) {
rule.state = stateMatch[1];
}
const interfaceMatch = line.match(/-i\s+(\S+)/);
if (interfaceMatch) {
rule.interface = interfaceMatch[1];
}
return rule;
}
function parseIptablesOutput(output: string): FirewallChain[] {
const chains: Map<string, FirewallChain> = new Map();
const lines = output.split("\n");
for (const line of lines) {
const trimmed = line.trim();
const policyMatch = trimmed.match(/^:(\S+)\s+(\S+)/);
if (policyMatch) {
const [, chainName, policy] = policyMatch;
chains.set(chainName, {
name: chainName,
policy: policy,
rules: [],
});
continue;
}
const rule = parseIptablesRule(trimmed);
if (rule) {
let chain = chains.get(rule.chain);
if (!chain) {
chain = {
name: rule.chain,
policy: "ACCEPT",
rules: [],
};
chains.set(rule.chain, chain);
}
chain.rules.push(rule);
}
}
return Array.from(chains.values());
}
function parseNftablesOutput(output: string): FirewallChain[] {
const chains: FirewallChain[] = [];
let currentChain: FirewallChain | null = null;
const lines = output.split("\n");
for (const line of lines) {
const trimmed = line.trim();
const chainMatch = trimmed.match(
/chain\s+(\S+)\s*\{?\s*(?:type\s+\S+\s+hook\s+(\S+))?/,
);
if (chainMatch) {
if (currentChain) {
chains.push(currentChain);
}
currentChain = {
name: chainMatch[1].toUpperCase(),
policy: "ACCEPT",
rules: [],
};
continue;
}
if (currentChain && trimmed.startsWith("policy ")) {
const policyMatch = trimmed.match(/policy\s+(\S+)/);
if (policyMatch) {
currentChain.policy = policyMatch[1].toUpperCase();
}
continue;
}
if (currentChain && trimmed && !trimmed.startsWith("}")) {
const rule: FirewallRule = {
chain: currentChain.name,
target: "",
protocol: "all",
source: "0.0.0.0/0",
destination: "0.0.0.0/0",
};
if (trimmed.includes("accept")) rule.target = "ACCEPT";
else if (trimmed.includes("drop")) rule.target = "DROP";
else if (trimmed.includes("reject")) rule.target = "REJECT";
const tcpMatch = trimmed.match(/tcp\s+dport\s+(\S+)/);
if (tcpMatch) {
rule.protocol = "tcp";
rule.dport = tcpMatch[1];
}
const udpMatch = trimmed.match(/udp\s+dport\s+(\S+)/);
if (udpMatch) {
rule.protocol = "udp";
rule.dport = udpMatch[1];
}
const saddrMatch = trimmed.match(/saddr\s+(\S+)/);
if (saddrMatch) {
rule.source = saddrMatch[1];
}
const daddrMatch = trimmed.match(/daddr\s+(\S+)/);
if (daddrMatch) {
rule.destination = daddrMatch[1];
}
const iifMatch = trimmed.match(/iif\s+"?(\S+)"?/);
if (iifMatch) {
rule.interface = iifMatch[1].replace(/"/g, "");
}
const ctStateMatch = trimmed.match(/ct\s+state\s+(\S+)/);
if (ctStateMatch) {
rule.state = ctStateMatch[1].toUpperCase();
}
if (rule.target) {
currentChain.rules.push(rule);
}
}
if (trimmed === "}") {
if (currentChain) {
chains.push(currentChain);
currentChain = null;
}
}
}
if (currentChain) {
chains.push(currentChain);
}
return chains;
}
export async function collectFirewallMetrics(
client: Client,
): Promise<FirewallMetrics> {
try {
const iptablesResult = await execCommand(
client,
"iptables-save 2>/dev/null",
15000,
);
if (iptablesResult.stdout && iptablesResult.stdout.includes("*filter")) {
const chains = parseIptablesOutput(iptablesResult.stdout);
const hasRules = chains.some((c) => c.rules.length > 0);
return {
type: "iptables",
status: hasRules ? "active" : "inactive",
chains: chains.filter(
(c) =>
c.name === "INPUT" || c.name === "OUTPUT" || c.name === "FORWARD",
),
};
}
const nftResult = await execCommand(
client,
"nft list ruleset 2>/dev/null",
15000,
);
if (nftResult.stdout && nftResult.stdout.trim()) {
const chains = parseNftablesOutput(nftResult.stdout);
const hasRules = chains.some((c) => c.rules.length > 0);
return {
type: "nftables",
status: hasRules ? "active" : "inactive",
chains,
};
}
return {
type: "none",
status: "unknown",
chains: [],
};
} catch {
return {
type: "none",
status: "unknown",
chains: [],
};
}
}
+154
View File
@@ -0,0 +1,154 @@
import type { Client } from "ssh2";
import { execCommand } from "./common-utils.js";
import type {
PortsMetrics,
ListeningPort,
} from "../../../types/stats-widgets.js";
function parseSsOutput(output: string): ListeningPort[] {
const ports: ListeningPort[] = [];
const lines = output.split("\n").slice(1);
for (const line of lines) {
const trimmed = line.trim();
if (!trimmed) continue;
const parts = trimmed.split(/\s+/);
if (parts.length < 5) continue;
const protocol = parts[0]?.toLowerCase();
if (protocol !== "tcp" && protocol !== "udp") continue;
const state = parts[1];
const localAddr = parts[4];
if (!localAddr) continue;
const lastColon = localAddr.lastIndexOf(":");
if (lastColon === -1) continue;
const address = localAddr.substring(0, lastColon);
const portStr = localAddr.substring(lastColon + 1);
const port = parseInt(portStr, 10);
if (isNaN(port)) continue;
const portEntry: ListeningPort = {
protocol: protocol as "tcp" | "udp",
localAddress: address.replace(/^\[|\]$/g, ""),
localPort: port,
state: protocol === "tcp" ? state : undefined,
};
const processInfo = parts[6];
if (processInfo && processInfo.startsWith("users:")) {
const pidMatch = processInfo.match(/pid=(\d+)/);
const nameMatch = processInfo.match(/\("([^"]+)"/);
if (pidMatch) portEntry.pid = parseInt(pidMatch[1], 10);
if (nameMatch) portEntry.process = nameMatch[1];
}
ports.push(portEntry);
}
return ports;
}
function parseNetstatOutput(output: string): ListeningPort[] {
const ports: ListeningPort[] = [];
const lines = output.split("\n");
for (const line of lines) {
const trimmed = line.trim();
if (!trimmed) continue;
const parts = trimmed.split(/\s+/);
if (parts.length < 4) continue;
const proto = parts[0]?.toLowerCase();
if (!proto) continue;
let protocol: "tcp" | "udp";
if (proto.startsWith("tcp")) {
protocol = "tcp";
} else if (proto.startsWith("udp")) {
protocol = "udp";
} else {
continue;
}
const localAddr = parts[3];
if (!localAddr) continue;
const lastColon = localAddr.lastIndexOf(":");
if (lastColon === -1) continue;
const address = localAddr.substring(0, lastColon);
const portStr = localAddr.substring(lastColon + 1);
const port = parseInt(portStr, 10);
if (isNaN(port)) continue;
const portEntry: ListeningPort = {
protocol,
localAddress: address,
localPort: port,
};
if (protocol === "tcp" && parts.length >= 6) {
portEntry.state = parts[5];
}
const pidProgram = parts[parts.length - 1];
if (pidProgram && pidProgram.includes("/")) {
const [pidStr, process] = pidProgram.split("/");
const pid = parseInt(pidStr, 10);
if (!isNaN(pid)) portEntry.pid = pid;
if (process) portEntry.process = process;
}
ports.push(portEntry);
}
return ports;
}
export async function collectPortsMetrics(
client: Client,
): Promise<PortsMetrics> {
try {
const ssResult = await execCommand(client, "ss -tulnp 2>/dev/null", 15000);
if (ssResult.stdout && ssResult.stdout.includes("Local")) {
const ports = parseSsOutput(ssResult.stdout);
return {
source: "ss",
ports: ports.sort((a, b) => a.localPort - b.localPort),
};
}
const netstatResult = await execCommand(
client,
"netstat -tulnp 2>/dev/null",
15000,
);
if (netstatResult.stdout && netstatResult.stdout.includes("Local")) {
const ports = parseNetstatOutput(netstatResult.stdout);
return {
source: "netstat",
ports: ports.sort((a, b) => a.localPort - b.localPort),
};
}
return {
source: "none",
ports: [],
};
} catch {
return {
source: "none",
ports: [],
};
}
}