Supplier price list import
Acme buys stock from a supplier called NordicGear, and NordicGear changes its prices from time to time. Each night the supplier posts a fresh price list, a simple spreadsheet-style file with one line per product, onto its own server. By the next morning Acme’s product catalogue should show those new prices. The supplier will not send the file to Acme, so Acme has to go and collect it: every night Acme logs in to the supplier’s server, downloads the file, checks it looks sensible, and updates the prices in its own database. The file is a full list each time, so collecting the same file twice does no harm. The case to guard against is a bad file, one that is cut short or garbled, because blindly loading it could wipe out good prices, so a file that fails its checks is set aside untouched and someone is told.
So the job runs itself once a night. Go to the supplier’s server, fetch the new price list, make sure it is complete and sane, and write the prices into Acme’s catalogue. Yesterday’s prices stay in force until a good file replaces them, and a file that fails its checks never touches the catalogue.
Here is how iQBus builds that. This is the polling consumer shape with a file source: a clock, a download, and a write. Everything inside the product rides a shared message backbone called the bus: parts of the flow publish messages onto it, and other parts subscribe to the ones they care about, so the pieces never call each other directly.
The way in is a receive port (an inbound endpoint that puts messages onto the bus) bound to the SFTP Listener (an adapter is the connector to one kind of outside system, here a partner file server). It polls the supplier’s server on its own interval, and as soon as the nightly file is there it downloads it, removes it from the server, and publishes it to the bus. There is no clock port and no subscription to wire up: the Listener is both the timer and the transfer.
The file enters the bus through that receive port’s own inbound leg, where a pipeline component (a small piece of custom code that reshapes a message in transit) does two jobs in one pass: it validates the CSV, and it disassembles it into the canonical PriceList JSON, stamping PriceList on the message it emits so the bus carries JSON rather than raw CSV. You do not write that component by hand: the AI Accelerator generates it from a plain-language description, and the working code is on this page as a fallback.
A second send port, bound to the SQL Caller, subscribes to the canonical message and calls a stored procedure that upserts the whole file in one MERGE into the product table. Because the file is a full refresh, the import is naturally idempotent: fetching the same file twice converges on the same prices.
{{Message.MessageType}} == "PriceList"
When it fails. Two ports can fail, the download and the merge, and both carry the same Exception Message Type, PriceImportFailed. On an error the in-flight message is republished under that type with its payload untouched, so the payload tells you which leg broke: a raw CSV body is a file that was fetched but failed its checks, a JSON body is a price list that would not merge. A third send port subscribes to it and writes each failure into a failures table operations works from, so the failure is handled, and the run does not suspend:
{{Message.MessageType}} == "PriceImportFailed"
Underneath that sits an Activity Notification, a per-Application email subscription that fires on any failed run. It is the safety net for what the exception route cannot foresee, including the route itself failing: it only tells a person, and the message it reports on stays suspended in tracking for replay. Step 5 builds both and explains why you want both. Either way yesterday’s prices stay in force, because nothing reaches the catalogue until a file has passed its checks.
Build it, step by step. The steps follow the integration itself, from the clock to the catalogue, not the order that is quickest to click through. Every Authentication, message type, pipeline and component is created from the dialog that first asks for it, so you never have to leave the port you are configuring to go and make something it needs. The product database objects get a step of their own just before the port that calls them.
| Setting | Value |
|---|---|
| Name | AcmePrices |
| Namespace | AcmePrices |
| Tracking | Enabled + Body |
| Description | Supplier price lists into the product catalogue |
Under the application’s Ports, create a receive port bound to the SFTP Listener. One port does the whole intake: it owns the timing, it makes the connection, and it publishes what it finds. If you have to step away mid-setup, Save Draft keeps the port under just its name; a draft does not exist to the runtime until the required fields are filled in and the port is saved properly.
| Setting | Value |
|---|---|
| Name | PriceListPickup |
| Application | AcmePrices (preset) |
| Port Type | Receive |
| Adapter | SFTP Listener |
| Adapter Message Type | SupplierPriceList, the type of the file coming in; Step 2 creates it |
| Exception Message Type | PriceImportFailed, created next, at this field |
| Way | One (the Listener publishes and moves on; there is nobody waiting for a reply) |
Every port has an Exception Message Type. When a run fails, the in-flight message has its type swapped to this one and is republished to the bus, payload untouched, so that another port can pick the failure up and do something with it; without it, the failed run simply suspends. The type does not exist yet, so create it at the field. Its Format describes what will most often be inside it, and on this port that is the picked-up CSV, so CSV is the honest answer even though the merge port in Step 3 will republish JSON under the same type. Step 4 gives the type its route.
| Setting | Value |
|---|---|
| Name | PriceImportFailed |
| Format | CSV |
| Sample data | empty; the body is whatever was in flight when the run failed, a raw file from this port or a canonical price list from the merge port |
| Promotions | none; CSV carries no promotions, and the route in Step 4 reads the whole body |
Open the adapter configuration (the button beside Adapter). At the Auth Config field, create the supplier’s SFTP credential in place; the dialog opens on top of the port and returns you to it when you save:
| Setting | Value |
|---|---|
| Name | SupplierSftp |
| Adapter | SFTP Listener (preset) |
| Definition | SFTP Password Authentication |
| Application | AcmePrices (preset) |
| Username (SFTP Password Settings, Credentials) | acme-pricefeed |
| Password (SFTP Password Settings, Credentials) | Ng!2026-pricelist-4b8d1e |
| Host Key Fingerprint (SFTP Password Settings, Server Identity) | SHA256:9Xf2kQpR7sT1uVwYzA3bC5dE8gH0jK2lM4nO6pQ8rS |
The username and password are whatever the supplier issued for your account; the values above are fine for the tutorial, and for a real feed use theirs and keep them out of email. The Host Key Fingerprint is optional in the dialog and belongs in every production configuration: when it is set, the adapter refuses to connect to a server whose host key does not match, which is the SFTP equivalent of certificate pinning. If the supplier rebuilds their server and the key changes, the pickup fails loudly instead of trusting a stranger. Get the fingerprint from the supplier over a channel that is not the SFTP connection itself, and expect to update it the day they tell you the server moved.
Back in the adapter configuration, describe the pickup. There is no operation to choose: a Listener polls, downloads and deletes, and that is the whole of it.
| Setting | Value |
|---|---|
| Auth Config Type | Static |
| Auth Config | SupplierSftp, just created |
| Host | sftp.nordicgear.example |
| Port | 22 |
| Polling Interval (seconds) | 300; five minutes. The supplier publishes overnight, and the file is collected within five minutes of appearing rather than at a time you nominate |
| Max Files Per Poll | 5; the supplier drops one file a night, and the headroom covers a night they drop a correction alongside it |
| Max Batch Size in Bytes | 10485760; ten megabytes, comfortably above a price list of a few thousand rows |
| Path | /outbound, the directory the supplier publishes into |
| File Mask | prices.csv; the exact name, so nothing else in that directory is ever taken |
| Include Subfolders | Off; the file is always at the top of /outbound |
Save the port. PriceListPickup is complete, stopped, and waiting; it is the last port you start, in Step 5. The next step continues on its inbound side.
Still inside PriceListPickup, move to the port’s inbound side, where the file the Listener picked up enters the bus. That side carries the required Adapter Message Type you named in Step 1, applied the instant the Listener releases the file to the next step inside the port, so the raw CSV is typed before any component sees it. The inbound side runs pipeline first, then map: transport-level work such as parsing comes before any reshaping.
NordicGear’s prices.csv as it arrives: a header line, then one line per product. Twelve products here; the real file runs to thousands, which is the whole reason a row-count floor is worth having:
sku,description,unit_price,currency,valid_from TP-4501,Trail Pack 45L,61.20,EUR,2026-09-08 TP-6501,Trail Pack 65L,84.50,EUR,2026-09-08 HX-0090,Hex Stove,18.75,EUR,2026-09-08 HX-0120,Hex Stove Wide,24.90,EUR,2026-09-08 GL-1240,Glacier Lantern,9.40,EUR,2026-09-08 GL-1260,Glacier Lantern Pro,14.60,EUR,2026-09-08 SB-2200,Summit Bag Two Season,112.00,EUR,2026-09-08 SB-2400,Summit Bag Four Season,168.00,EUR,2026-09-08 FL-0310,Fjord Flask 1L,21.30,EUR,2026-09-08 FL-0320,Fjord Flask 2L,27.80,EUR,2026-09-08 RD-7700,Ridge Poles Pair,39.95,EUR,2026-09-08 RD-7720,Ridge Poles Carbon,74.25,EUR,2026-09-08
Select the type for it at the Adapter Message Type field, or create it there. Paste the file above into Sample data: it has no runtime effect, but it documents the shape for whoever opens the type later, and it is the same file you will place on the server in Step 6.
| Setting | Value |
|---|---|
| Name | SupplierPriceList |
| Format | CSV |
| Sample data | the prices.csv above |
| Promotions | none; only XML and JSON types carry promotions, and there is nothing to route on before the file has been checked anyway |
Saving the type fills the Adapter Message Type row that Step 2 left open on this port.
Next comes the response Pipeline. A port never selects a component directly; it selects a pipeline, a named, ordered list of pipeline components that also holds the default value for each component’s configuration property. Create the pipeline at the Pipeline field, and the component from the pipeline’s Components list; the component is where the code lives, and it is the one iQBus artifact in this tutorial you do not have to write yourself.
You do not have to write this component by hand. Open the AI Accelerator, describe it in plain language, and let it generate the C# for you to review and save. It knows how iQBus pipeline components work, the base class, how a component returns messages and types each one, and how a failed component stops a run, but it sees nothing of your Application: not the ports, not the message types you have created, nothing but the text you give it. So the prompt carries everything the component needs: the file it reads, the JSON it must emit, the four properties by name, and what counts as a bad file. A prompt like this is enough:
Build a pipeline component named PriceListDisassembler that validates a supplier price list CSV and turns it into one canonical JSON message.
The inbound body is plain text: a header line, then one line per product. Give the component four required configuration properties, none of them with a default in code, so a missed setting stops the file instead of waving it through. ExpectedHeader is the header line the file must start with, matched exactly. MinRows is the smallest number of product lines a file may have and still be trusted. Supplier is the supplier name to stamp on the result. EmittedMessageType is the message type to put on the message the component emits.
Validate before parsing anything, and fail the component with a clear error message on any of these: the file is empty, the first line does not equal ExpectedHeader, there are fewer product lines than MinRows, a product line has fewer than five columns, or a price does not parse as a decimal greater than zero. Failing the component is the point, a file that does not pass must not reach the database.
If the file passes, emit exactly one message whose body is the whole price list as JSON, and stamp it with the message type named by EmittedMessageType. Match the columns to the JSON fields by meaning: the product code, the unit price as a number, and the currency go into one entry per product; the valid-from date of the first product line becomes the list's own valid-from; the supplier name comes from the Supplier property, not from the file. Leave the description column out, the catalogue does not price on it. The three field names must be exactly sku, price and currency inside the entries, and supplier, validFrom and items at the top, because the merge procedure reads them by those names.
The file it reads looks like this (yours will carry different products and a different date, but the same columns in the same order):
sku,description,unit_price,currency,valid_from
TP-4501,Trail Pack 45L,61.20,EUR,2026-09-08
TP-6501,Trail Pack 65L,84.50,EUR,2026-09-08
HX-0090,Hex Stove,18.75,EUR,2026-09-08
The single message it emits must look like this:
{
"supplier": "NordicGear",
"validFrom": "2026-09-08",
"items": [
{ "sku": "TP-4501", "price": 61.20, "currency": "EUR" },
{ "sku": "TP-6501", "price": 84.50, "currency": "EUR" },
{ "sku": "HX-0090", "price": 18.75, "currency": "EUR" }
]
}The AI Accelerator writes a first draft, not a finished component: read what it produced, check that the four properties are named ExpectedHeader, MinRows, Supplier and EmittedMessageType exactly, since the pipeline table below sets them by those names, and confirm in Step 6 that a good file produces one JSON message in tracking and a short file produces none. If you would rather start from working code, expand it here to read or copy.
Show the full component code (C#), expand to read or copy
The component splits the body into non-empty lines, checks the header and the row count before it parses a single price, then walks the rows and builds one anonymous object per product. A row that is short or carries a price that is not a positive decimal fails the whole file rather than being skipped, because a partially parsed price list is more dangerous than none. The result is serialized once and returned as a single PipelineMessage stamped with the configured type. Every property is marked required with no default in code, which is why the pipeline has to supply all four.
using System.ComponentModel.DataAnnotations; using System.Text.Json; using CC.iQBus.Pipelines.Domain.Models.PipelineComponents; public sealed class PriceListDisassemblerConfig { // The header line the file must start with, matched exactly. [Required] public string ExpectedHeader { get; set; } = string.Empty; // The smallest number of product lines a trustworthy file has. [Range(1, int.MaxValue)] public int MinRows { get; set; } // Stamped on the result; the file does not name its own supplier. [Required] public string Supplier { get; set; } = string.Empty; // Message Type assigned to the message this component emits. [Required] public string EmittedMessageType { get; set; } = string.Empty; } public sealed class PriceListDisassembler : PipelineComponentBase<PriceListDisassemblerConfig> { public override string Name => "PriceListDisassembler"; protected override Task<PipelineComponentOutput> ExecuteAsync( PipelineComponentInput input, PriceListDisassemblerConfig config, CancellationToken cancellationToken) { // 1. Read the raw CSV body the SFTP download handed in. var lines = input.Body .Split('\n') .Select(l => l.TrimEnd('\r')) .Where(l => l.Length > 0) .ToList(); // 2. Validate before parsing a single price. if (lines.Count == 0 || lines[0] != config.ExpectedHeader) return Fail("price list header missing or unexpected"); if (lines.Count - 1 < config.MinRows) return Fail($"price list has {lines.Count - 1} rows, " + $"expected at least {config.MinRows}"); // 3. Disassemble the rows into the canonical PriceList shape. var items = new List<object>(); string? validFrom = null; foreach (var row in lines.Skip(1)) { var cols = row.Split(','); if (cols.Length < 5 || !decimal.TryParse(cols[2], out var price) || price <= 0m) return Fail($"invalid price row: {row}"); validFrom ??= cols[4]; items.Add(new { sku = cols[0], price, currency = cols[3] }); } var json = JsonSerializer.Serialize(new { supplier = config.Supplier, validFrom, items }); // 4. Emit one JSON message and stamp it with the type the pipeline // names, so the bus carries PriceList and never the raw CSV. return Task.FromResult(new PipelineComponentOutput { Success = true, Messages = [new PipelineMessage { Body = json, MessageType = config.EmittedMessageType }] }); static Task<PipelineComponentOutput> Fail(string msg) => Task.FromResult(new PipelineComponentOutput { Success = false, ErrorMessage = msg }); } }
| Setting | Value |
|---|---|
| Name | PriceListDisassembler |
| Code | the C# the AI Accelerator produced, or the collapsed code above |
One message for the whole file, with the description column dropped and the supplier name stamped from configuration. Three products are shown; the rest of the file follows the same shape:
{
"supplier": "NordicGear",
"validFrom": "2026-09-08",
"items": [
{ "sku": "TP-4501", "price": 61.20, "currency": "EUR" },
{ "sku": "TP-6501", "price": 84.50, "currency": "EUR" },
{ "sku": "HX-0090", "price": 18.75, "currency": "EUR" }
]
}The component stamps a type name on what it emits, and that type has to exist for the catalogue port in Step 4 to subscribe to it. This is the one artifact in this tutorial that no dialog offers to create for you, and the reason is worth knowing: inline creation happens at a field that references an artifact, and nothing here does. The pipeline property below takes a plain string, and a subscription expression is text. So create this one under the application’s Message types, the same way a schema is made in its own dialog, then come back. Paste the JSON above into Sample data: nothing in this flow needs a promotion, but the sample is what Generate Path reads from if a later flow ever routes on a field in here, and it documents the shape for whoever opens the type next.
| Setting | Value |
|---|---|
| Name | PriceList |
| Format | JSON |
| Sample data | the PriceList JSON above |
| Promotions | none; nothing routes on a value inside the list, and the merge port sends the whole body as one parameter |
Back in the pipeline with the component listed, set all four of its properties. None has a default in code, which is exactly why a missed setting stops the file instead of waving it through. EmittedMessageType is plain text, the name of the type you just created, spelled the same way:
| Setting | Value |
|---|---|
| Name | PriceListImportPipeline |
| Components | PriceListDisassembler, just created |
| ExpectedHeader (default for the component) | sku,description,unit_price,currency,valid_from |
| MinRows (default for the component) | 10, a floor low enough never to reject a real file and high enough to catch a truncated one; set it from the smallest file the supplier has ever sent, not from the largest |
| Supplier (default for the component) | NordicGear |
| EmittedMessageType (default for the component) | PriceList |
| Setting | Value |
|---|---|
| Pipeline | PriceListImportPipeline, just created |
| Setting | Value |
|---|---|
| Map Assignment | None; the component already emitted the canonical shape, and there was never a map to run here anyway because a CSV body cannot be a map source |
Save the port. PriceListPickup is complete: every tick it acts on becomes either one PriceList message on the bus, or a failure, and never a half-loaded catalogue.
The product catalogue is the system you are integrating with, not part of iQBus, so it is set up by hand rather than asked of the AI Accelerator. Create the table and the procedure the send port calls in Step 4. OPENJSON reads the canonical PriceList body directly, so the whole file lands in one MERGE, and the product code is the primary key, which is what makes a repeated import converge instead of duplicating.
CREATE TABLE dbo.Products ( Sku VARCHAR(40) NOT NULL PRIMARY KEY, Price DECIMAL(18,2) NOT NULL, Currency CHAR(3) NOT NULL, UpdatedAt DATETIME2 NOT NULL CONSTRAINT DF_Prod_Upd DEFAULT SYSUTCDATETIME() );
CREATE OR ALTER PROCEDURE dbo.MergePriceList @PriceListJson NVARCHAR(MAX) AS BEGIN SET NOCOUNT ON; MERGE dbo.Products AS tgt USING ( SELECT sku, price, currency FROM OPENJSON(@PriceListJson, '$.items') WITH ( sku VARCHAR(40) '$.sku', price DECIMAL(18,2) '$.price', currency CHAR(3) '$.currency' ) ) AS src ON tgt.Sku = src.sku WHEN MATCHED THEN UPDATE SET Price = src.price, Currency = src.currency, UpdatedAt = SYSUTCDATETIME() WHEN NOT MATCHED THEN INSERT (Sku, Price, Currency) VALUES (src.sku, src.price, src.currency); END;
Under the application’s Ports, create a send port bound to the SQL Caller, one-way. Rather than being told what to deliver, a send port states which messages it wants; this one wants every canonical price list the fetch port publishes. The Exception Message Type already exists from Step 2, so this time it is a selection:
| Setting | Value |
|---|---|
| Name | PriceUpsert |
| Application | AcmePrices (preset) |
| Port Type | Send |
| Adapter | SQL Caller |
| Adapter Message Type | none; one-way, so nothing comes back to be typed |
| Exception Message Type | PriceImportFailed, from Step 2 |
| Way | One (the merge either succeeds or raises, and there is no result the flow needs) |
| Subscription | {{Message.MessageType}} == "PriceList" |
| Setting | Value |
|---|---|
| Open Variables | empty; the parameter below reads the message itself, so nothing needs carrying in |
| Map | None; the body is already the canonical shape the procedure reads |
| Pipeline | empty; there is nothing to frame or compress on the way to a database |
| Close Variables | empty; nothing to compute at the last moment |
Open the adapter configuration. At the Auth Config field, create the catalogue database credential in place, the same way you created the supplier’s SFTP credential:
| Setting | Value |
|---|---|
| Name | ProductDbSql |
| Adapter | SQL Caller (preset) |
| Definition | SQL Server Connection |
| Application | AcmePrices (preset) |
| Connection String (Database Config) | one of the two below |
The SQL Caller connects from where iQBus runs, so two things decide the string: how iQBus reaches the server and how the server identifies iQBus. Reachability first: an Azure SQL database on a public endpoint needs iQBus’s outbound address allowed through the server firewall; one behind a private endpoint has to sit in, or be peered with, the iQBus network; an on-premises SQL Server is reached over a VPN or ExpressRoute link, and its hostname has to resolve from the iQBus side. Then the identity, which is what the two strings differ in. iQBus supports SQL authentication and managed identity; it does not support Entra ID interactive sign-in, and putting an Entra account’s username and password into a connection string is strongly discouraged, so neither appears here. Replace the placeholders in angle brackets with your own values:
| Method | Connection String | Use it when |
|---|---|---|
| SQL authentication | Server=tcp:<server>.database.windows.net,1433;Database=<database>;User ID=<login>;Password=<password>;Encrypt=True;TrustServerCertificate=False; | the simplest to start with, and the only option for a SQL Server that is not Entra-integrated; the password lives in this Authentication, encrypted at rest |
| Managed identity | Server=tcp:<server>.database.windows.net,1433;Database=<database>;Authentication=Active Directory Managed Identity;Encrypt=True; | no secret at all: the iQBus deployment’s own Azure identity is granted access on the database (CREATE USER [<identity>] FROM EXTERNAL PROVIDER, then the roles it needs); the strongest choice for production |
For an on-premises server, swap the Server= value for its hostname or IP and port and use SQL authentication. For this tutorial, SQL authentication against a database whose firewall lets iQBus in is enough. Whichever you pick, the login needs EXECUTE on dbo.MergePriceList, and in Step 5 on dbo.RecordPriceImportFailure as well, because the same Authentication serves both ports.
Back in the adapter configuration, set the command and its parameter together. The procedure from Step 3 takes the whole price list as one value, so there is a single Input Parameters row and no promotion is involved:
| Setting | Value |
|---|---|
| Auth Config | ProductDbSql, just created |
| Command Type | Stored Procedure |
| Command Text | dbo.MergePriceList, from Step 3 |
| Parameter | Value |
|---|---|
| PriceListJson | {{Message.Body}} |
Adapter parameters are plain strings that bind {{…}} tokens but never evaluate a body path themselves. {{Message.Body}} is the whole message, which is exactly what is wanted here: the procedure does the reading with OPENJSON on the database side. A flow that needed one field out of the list instead would have to promote it on the message type first, the way the order intake tutorial promotes its three values. Save the port.
Two things can go wrong in this flow, and you already named them both PriceImportFailed: the download or its checks fail, so the raw file is republished under that type, or the merge fails, so the canonical price list is republished under it. Right now nothing subscribes to that type, so a failure would go nowhere. This step configures failure handling twice, because the two mechanisms do different jobs.
The route needs somewhere to put the failure, and for a price import that somewhere had better hold the file itself: a bad night is only diagnosable if you can read what arrived. Operations already lives in the product database, so the table goes there for this tutorial; in production you might prefer a database of your own, precisely so a catalogue outage does not take the failure log with it. Like Step 3, this is scaffolding for an external system, so it is supplied as SQL rather than asked of the AI Accelerator. The procedure classifies the failure from its payload: a canonical price list is JSON, and a rejected supplier file is not, so ISJSON tells the two legs apart without the flow having to label them.
CREATE TABLE dbo.PriceImportFailures ( FailureId BIGINT IDENTITY(1,1) NOT NULL PRIMARY KEY, Leg VARCHAR(10) NOT NULL, -- Fetch or Merge Payload NVARCHAR(MAX) NULL, -- the rejected file, or the list that would not merge FailedAt DATETIME2 NOT NULL CONSTRAINT DF_PriceFail_At DEFAULT SYSUTCDATETIME(), ResolvedAt DATETIME2 NULL -- operations sets this once the import is re-run );
CREATE OR ALTER PROCEDURE dbo.RecordPriceImportFailure @Payload NVARCHAR(MAX) AS BEGIN SET NOCOUNT ON; INSERT INTO dbo.PriceImportFailures (Leg, Payload) VALUES (CASE WHEN ISJSON(@Payload) = 1 THEN 'Merge' ELSE 'Fetch' END, @Payload); END;
Under the application’s Ports, create one more send port on the SQL Caller, one-way: it subscribes to the failure type and calls the procedure above. It reuses the ProductDbSql Authentication from Step 4, so at the Auth Config field this is a selection, not a creation. Leave its own Exception Message Type empty: a route that republished its own failures as PriceImportFailed would pick them straight back up, and that loop is exactly the case the Activity Notification below exists for.
| Setting | Value |
|---|---|
| Name | FailedImportToDeadLetter |
| Application | AcmePrices (preset) |
| Port Type | Send |
| Adapter | SQL Caller |
| Adapter Message Type | none; one-way |
| Exception Message Type | empty, on purpose; see the paragraph above |
| Way | One |
| Subscription | {{Message.MessageType}} == "PriceImportFailed" |
| Setting | Value |
|---|---|
| Open Variables | empty; the parameter reads the failed message itself |
| Map | None; the payload is stored exactly as it failed, which is the point of keeping it |
| Pipeline | empty; nothing to pre-process |
| Close Variables | empty; nothing to compute |
| Setting | Value |
|---|---|
| Auth Config | ProductDbSql, from Step 4 |
| Command Type | Stored Procedure |
| Command Text | dbo.RecordPriceImportFailure |
| Parameter | Value |
|---|---|
| Payload | {{Message.Body}} |
Save the port. From now on a rejected file lands as a Fetch row with the file itself in Payload, ready for someone to read the first line and see what the supplier changed, and the run completes as handled.
Under the application’s Activity Notifications, create the notification. With Subscription on All and On Error Only on, it fires for any failed run in the Application, whichever port failed and whether or not you foresaw it:
| Setting | Value |
|---|---|
| Name | PriceImportErrors |
| Application | AcmePrices (preset) |
| On Error Only | On |
| Subscription | All |
| Mode | On Event |
| Recipients | one row: Name Operations, Email ops@acme.example |
| Is Enabled | Enabled |
Nothing in this flow speaks HTTP, so there is no API client to set up and no mock server to stand anything in for. The two systems are a file server and a database, and you test it with the tools you would use on either one. Have these ready, because each one is something the test cannot prove without:
| What | Why the test needs it |
|---|---|
| An SFTP client, WinSCP, FileZilla, or the sftp command line; any of them works the same way | to place the test file in /outbound on the supplier server, and later to take it away again. Sign in with the same username and password you put in SupplierSftp: if your client cannot read that directory, neither can the port |
| A SQL client, SQL Server Management Studio or Azure Data Studio | to read dbo.Products and dbo.PriceImportFailures. Connect as the login in ProductDbSql and confirm it has EXECUTE on both procedures, or the merge will fail for a reason that has nothing to do with the file |
| The operations mailbox you put in PriceImportErrors, and the platform Email notification configuration already done | the second break-it test below is entirely about an email arriving. Without the platform configuration nothing sends, and a silent mailbox will look like a passing test |
There is nothing to deploy, everything you configured is already saved and live, because iQBus applies changes immediately. To bring the flow online you start the send ports first, then the receive port, and among the send ports, the ones furthest downstream first: start FailedImportToDeadLetter, then PriceUpsert, then PriceListPickup, and last the pickup port, PriceListPickup. Starting each subscriber before its producer means no price list is published before the catalogue port is ready to take it, and nothing is collected from the supplier before the flow that consumes it exists. (To take the flow offline, stop the pickup port first, for the same reason.) Tracking is already Enabled + Body from the Application, so every run you make now is recorded step by step with its payload.
Five minutes between polls is right for a nightly feed and slow when you are watching a run. Change one setting on PriceListPickup, Restart the port from the ports list to reset its timer, and you get a poll every ten seconds:
| Setting | Test value | Restore to |
|---|---|---|
| Polling Interval (seconds) | 10 | 300 |
Upload the twelve-row prices.csv from Step 2 to /outbound/prices.csv on the supplier server, then wait for the next poll. Open Tracking, find the run the pickup started, and step through it, opening each message body as you go:
| Step in Tracking | What to confirm |
|---|---|
| PriceListPickup polls | the poll found prices.csv under /outbound and picked it up; check the supplier directory and confirm the file is gone |
| The file is typed | the body is the raw CSV, header line and twelve product lines, typed SupplierPriceList at the adapter |
| Pipeline PriceListImportPipeline | one message comes out, typed PriceList: JSON, twelve entries, the description column gone, supplier reading NordicGear from the pipeline’s own setting rather than from the file |
| Publish | the PriceList is on the bus and PriceUpsert picked it up |
| SQL Caller | the call to dbo.MergePriceList succeeded with the whole body as PriceListJson |
Then check the catalogue itself: SELECT COUNT(*) FROM dbo.Products returns 12, and SELECT * FROM dbo.Products WHERE Sku = 'TP-4501' shows 61.20 and EUR.
Leave the file where it is and let another tick pass. The row count stays at 12 and every price is unchanged; only UpdatedAt moves, because the MERGE matched every product and rewrote the same values. That is the whole reason a full-refresh feed is comfortable to run: a doubled tick, a replayed run, or a re-import after a fix all converge on the same catalogue. Now edit one price in the file, upload it again, and confirm that product alone changes.
This is the failure the scenario was written around: a bad file must not reach the catalogue. Cut prices.csv down to the header and five product lines, upload it over the good one, and wait for the tick. The download succeeds, and then the component rejects the file, because five rows is below the MinRows floor of 10. Watch what happens next in tracking: the run fails inside PriceListPickup, the raw file is republished as PriceImportFailed, FailedImportToDeadLetter picks it up and calls the procedure, and the run finishes as handled, not suspended. In the database, SELECT Leg, FailedAt, Payload FROM dbo.PriceImportFailures shows one Fetch row holding the five-line file, and the check that matters most, SELECT COUNT(*), MAX(UpdatedAt) FROM dbo.Products, is untouched: twelve products, still at yesterday’s timestamp. No email is sent, because nothing was left unhandled. Put the good twelve-row file back and let a tick clear it.
Worth trying the other rejections while you are here, because they are the ones a real supplier will hand you: rename a header column, or set one price to 0.00, and confirm each lands as its own Fetch row with the component’s error message on the failed run in tracking.
Now take away something the route itself depends on: stop the product database, or temporarily revoke EXECUTE on both dbo.MergePriceList and dbo.RecordPriceImportFailure, and wait for the tick with the good file in place. The download and the checks both succeed, the merge fails, the price list is republished as PriceImportFailed, and the route tries to write it to the same database, which fails too. This is the case no route can handle, and it is where the notification earns its keep: the operations mailbox gets the PriceImportErrors email from Step 5, and the failed run sits suspended in tracking with the whole price list intact, available for replay once the database is back. Restore the database or the permission, replay the suspended run from tracking, and confirm it completes and the prices land.