Description
Overview
This SWIFT code data extraction automation workflow enables efficient scraping and structured data collection from a banking directory website. Designed as an orchestration pipeline, it targets developers and data engineers needing reliable, no-code integration to gather comprehensive financial institution information. The workflow initiates via a manual trigger node and leverages HTTP request and HTML extraction nodes to systematically parse country and bank details.
Key Benefits
- Systematically extracts SWIFT codes and related bank data with batch-controlled pagination.
- Integrates country normalization to standardize ISO codes for consistent geographic referencing.
- Implements local HTML caching to minimize redundant network requests and optimize throughput.
- Structures extracted data into MongoDB documents for scalable storage and querying.
Product Overview
This automation workflow begins with a manual trigger, which initiates the creation of a local caching directory to store HTML files. It fetches the main country browsing page by issuing an HTTP GET request, then extracts country-specific URLs from an ordered list on the page using a CSS selector. Each country URL is processed in sequence through a batch mechanism, ensuring controlled scraping without overload.
For each country, the workflow normalizes the country name to an ISO code using a geographic normalization tool. Pagination is handled by storing the current page in workflow static data, enabling iterative retrieval of multiple pages per country. The workflow checks for cached HTML files before making fresh HTTP requests, reducing redundant downloads. Bank data such as names, SWIFT codes, cities, and branches are extracted from table cells within the HTML content.
Extracted data is transformed into structured documents with timestamps and stored in a MongoDB collection for persistence. Pagination logic detects “next” page links to continue scraping until no further pages exist, then moves on to the next country in the batch. Error handling relies on default platform behaviors; there are no explicit retry or backoff strategies configured.
Features and Outcomes
Core Automation
This no-code integration pipeline processes country URLs in single-item batches, applying geographic normalization and paginated scraping. The core logic uses HTML extraction nodes to parse structured bank details and handles pagination through static workflow state.
- Sequential batch processing ensures controlled scraping per country URL.
- Single-pass evaluation per page with conditional pagination continuation.
- Deterministic document preparation for consistent MongoDB insertion.
Integrations and Intake
The workflow integrates HTTP request nodes to fetch web pages and a geographic normalization tool to convert country URLs into ISO codes. Authentication is not required for the source website, while MongoDB credentials are used for data storage. Input payloads comprise URLs and query parameters embedded within static workflow data.
- HTTP requests retrieve main and paginated country pages for scraping.
- Geographic normalization API standardizes country identifiers.
- MongoDB connection enables direct insertion of structured bank documents.
Outputs and Consumption
The workflow outputs structured JSON documents representing bank entities, including ISO country codes, bank names, SWIFT codes, cities, and branches. Data is inserted asynchronously into a MongoDB collection. No synchronous client response or external delivery endpoints are configured.
- Structured JSON documents with geographic and banking metadata.
- Asynchronous database insertion with timestamped records.
- Local HTML cache files stored to optimize repeated workflow runs.
Workflow — End-to-End Execution
Step 1: Trigger
The workflow initiates manually via a dedicated manual trigger node. This controlled start allows users to execute the scraping process on demand.
Step 2: Processing
The workflow creates a local directory for caching scraped HTML files. It then fetches the main country list page and extracts country URLs. These URLs are transformed into individual items and processed in batches of one to maintain sequential scraping order.
Step 3: Analysis
Each country URL is normalized to an ISO code using a geographic normalization integration. The workflow manages page URLs through static data and checks for cached HTML files to avoid redundant downloads. Bank data is extracted from HTML tables with selectors targeting bank names, SWIFT codes, cities, and branches. Pagination is handled by detecting “next” page links, enabling iterative scraping until all pages are processed.
Step 4: Delivery
After data extraction and preparation, bank records are inserted into a MongoDB collection asynchronously. The workflow updates pagination state for subsequent scraping or proceeds to the next country batch, maintaining continuous data ingestion without external synchronous responses.
Use Cases
Scenario 1
A financial data analyst requires comprehensive SWIFT code listings across multiple countries. This automation workflow extracts and normalizes country-specific bank data, enabling the analyst to access up-to-date structured records stored in a MongoDB database for further analysis.
Scenario 2
A compliance officer needs to verify bank branch information for cross-border transactions. The orchestration pipeline systematically scrapes bank names, branches, and SWIFT codes, providing accurate, normalized data ready for integration into compliance systems.
Scenario 3
An application developer wants to maintain a local repository of international banking codes. The no-code integration workflow incrementally caches HTML pages and updates MongoDB records, ensuring a reliable source of SWIFT data without manual intervention.
How to use
After importing this workflow into n8n, users must configure MongoDB credentials for database access and uProc API credentials for country normalization. The workflow runs manually via the trigger node, fetching and processing data sequentially. Cached HTML files are stored locally to improve performance on repeated runs. Results appear as inserted documents in MongoDB, accessible for querying or further processing.
Comparison — Manual Process vs. Automation Workflow
| Attribute | Manual/Alternative | This Workflow |
|---|---|---|
| Steps required | Multiple manual web navigations and copy-paste operations. | Single execution triggers sequential scraping and data insertion. |
| Consistency | Prone to human error and incomplete data capture. | Deterministic extraction with normalized data and batch control. |
| Scalability | Limited by manual throughput and time constraints. | Scales via batch processing and asynchronous database writes. |
| Maintenance | High maintenance due to frequent manual updates and errors. | Low maintenance with automated pagination and cached HTML reuse. |
Technical Specifications
| Environment | n8n automation platform with MongoDB integration |
|---|---|
| Tools / APIs | HTTP Request, HTML Extract, uProc geographic normalization, MongoDB node |
| Execution Model | Manual trigger, batch sequential processing, asynchronous database insertion |
| Input Formats | HTTP responses (HTML), URL strings |
| Output Formats | Structured JSON documents stored in MongoDB |
| Data Handling | Local HTML caching, transient data transformation, timestamped records |
| Known Constraints | Relies on external website availability and structure consistency |
| Credentials | MongoDB credentials, uProc API key |
Implementation Requirements
- Valid MongoDB credentials with write access to the target collection.
- uProc API key configured for country normalization services.
- File system access permissions for local HTML cache directory creation.
Configuration & Validation
- Verify MongoDB credentials by testing connection and insert permissions.
- Confirm uProc API key is active and returns correct ISO country codes.
- Run the manual trigger and monitor logs for successful HTTP requests and data extraction.
Data Provenance
- Started by manualTrigger node initiating workflow.
- Country URLs extracted by HTML Extract node from main browsing page.
- Bank records prepared in function node and inserted via MongoDB node.
FAQ
How is the SWIFT code data extraction automation workflow triggered?
The workflow is triggered manually via the “On clicking ‘execute'” manual trigger node, allowing controlled execution on demand.
Which tools or models does the orchestration pipeline use?
The pipeline uses HTTP Request nodes to fetch web pages, an HTML Extract node to parse data, a uProc integration for geographic normalization, and a MongoDB node for data storage.
What does the response look like for client consumption?
The workflow outputs structured JSON documents containing ISO country codes, bank names, SWIFT codes, cities, and branches. These are stored asynchronously in MongoDB for downstream use.
Is any data persisted by the workflow?
Yes, extracted data is persisted as JSON documents in a MongoDB collection, while HTML pages are cached locally as files for reuse.
How are errors handled in this integration flow?
No explicit error handling or retry mechanisms are configured; error handling relies on n8n platform defaults for node execution.
Conclusion
This SWIFT code data extraction automation workflow provides a deterministic, batch-driven method to scrape and organize banking information from a public directory. It ensures consistent data normalization through geographic coding and maintains efficient operation by caching HTML content locally. The workflow depends on the availability and structure of the external website, which may require monitoring for changes. By automating pagination and data insertion, it reduces manual labor and supports scalable data management within MongoDB environments.
Reviews
There are no reviews yet.