Wrap import, export, and the e-document importer

First piece of the last roadmap item. Five routes: `POST /export`,
`POST /preimport`, `POST /import`, `POST /import_json`, and
`PUT /edocument/upload`.

A CSV import is two calls, and only that form is wrapped. `preimport`
uploads one file per entity type as `files[<entity_type>]` — a different
field name from the `documents[]` the `/{id}/upload` routes take — and
answers with a hash plus the column mapping the server suggests. `import`
then starts the ingest from that hash, as plain JSON: with a hash present
`ImportController::import` never looks at uploaded files.

The route does also accept the files and the mapping together in one
multipart request. That form is left out on purpose. A mapping is a nested
structure, and bracket-encoding nested structures into form fields is the
thing this crate avoids by sending JSON everywhere else — the same reason
`COVERAGE.md` gives for diverging from the PHP SDK's `form_params`.

`upload_edocument` is not a document attachment, which is why it is not an
`upload_*` method and why the roadmap had already split it out:
`ExpenseController::edocument` parses each file and creates *expenses*. It
shares the `_method=PUT` spoof with the upload routes, since it is likewise
registered as a `PUT`, but its reply has no `data` envelope — hence the new
`multipart_unwrapped` helper, which dispatches a form built at the call site
rather than imposing one shape on all four import routes.

Two new models, for the two shapes that are stable and that are the point of
the call: `ExportJob` (the collection URL, which expires in one hour, or
five past 10,000 activities) and `ImportPreview` (the hash). Their
`available`/`headers`/`hints` payloads stay `Value` — those come from each
entity's own `importable()` map and are undocumented, so a concrete type
would be a hard decode error where `Value` is lossless.

All five queue work and answer before it happens, so a 200 means accepted.
Failed rows are reported by email, not in the response, and `import_json`
overwrites the company it is pointed at.

`/migration/start` stays unwrapped and is now listed on its own: it is
driven by the v4 migration tool rather than by an SDK caller.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
2026-10-24 23:17:36 -04:00
co-authored by Claude Opus 5
parent e6e41fe5c6
commit c4f31743f0
9 changed files with 412 additions and 15 deletions
+12
View File
@@ -9,6 +9,18 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Added
- Import and export, five methods: `export`, `preimport`, `import`,
`import_json`, and `upload_edocument`. A CSV import is two calls —
`preimport` uploads the files as `files[<entity_type>]` and returns the
column mapping the server suggests, then `import` starts the ingest from
the returned `hash`. The route's one-shot form (files and mapping in a
single multipart request) is deliberately not wrapped: a mapping is nested,
and bracket-encoding nested structures is what this crate avoids by sending
JSON. `upload_edocument` imports expenses from e-documents and is not a
document attachment, which is why it is not an `upload_*` method. Adds
`ExportJob` (the export's collection URL expires in 1–5 hours) and
`ImportPreview`. All of these queue work, so a success means accepted, not
done.
- Invitation-key downloads, eight methods:
`download_invoice_by_invitation_key` and
`download_e_invoice_by_invitation_key`, plus the same pair for quotes,
+2 -2
View File
@@ -337,9 +337,9 @@ changes, doc tweaks) does not need an entry.
## Coverage
**All 34 of the API's `Route::resource` groups are wrapped**, plus `users`
(which isn't one). 333 methods: 30 resources with CRUD + bulk, 3 with CRUD
(which isn't one). 338 methods: 30 resources with CRUD + bulk, 3 with CRUD
only, 4 with entity actions, 4 with invitation-key downloads, 16 with
document uploads, and 69 hand-written.
document uploads, and 74 hand-written.
Plus the non-CRUD surface: documents, entity actions and PDF downloads for
the four action-capable resources, their invitation-key PDF and e-document
+37 -7
View File
@@ -18,7 +18,7 @@ declares 34 `Route::resource` groups and 274 explicit route declarations
> exists or what it returns.
**All 34 `Route::resource` groups are now wrapped**, plus `users` (which is
not a resource group). 333 methods in total.
not a resource group). 338 methods in total.
## Resources with full CRUD + bulk
@@ -168,6 +168,38 @@ controllers return that directly rather than raising
for model` text a missing record produces elsewhere, but the status is the
same 400.
## Import and export
| Route | Method | Returns |
| --- | --- | --- |
| `POST /export` | `export` | `ExportJob` |
| `POST /preimport` | `preimport` | `ImportPreview` |
| `POST /import` | `import` | `MessageResponse` |
| `POST /import_json` | `import_json` | `MessageResponse` |
| `PUT /edocument/upload` | `upload_edocument` | `MessageResponse` |
A CSV import is two calls. `preimport` uploads the files — one per entity
type, as `files[<entity_type>]` — and answers with a `hash` plus the column
mapping the server suggests; `import` then starts the ingest from that hash.
The server holds the files against the hash for 20 minutes.
The route also accepts the files and the mapping together in one multipart
request. That form is deliberately not wrapped: a mapping is a nested
structure, and bracket-encoding nested structures into form fields is exactly
what this crate avoids by sending JSON (see [the PHP SDK
comparison](#compared-with-the-official-php-sdk)). With a `hash` present the
controller never looks at uploaded files, so `import` is plain JSON.
`upload_edocument` is *not* a document attachment, which is why it is not one
of the `upload_*` methods: `ExpenseController::edocument` parses each file and
creates **expenses**. Like the `/{id}/upload` routes it is registered as `PUT`
and so goes out as a `POST` with `_method=PUT`; unlike them the reply has no
`data` envelope.
All five queue background work, so a 200 means accepted, not finished. Rows
that fail to import are reported by email. `import_json` restores a company
from an `export` backup and **overwrites the target company**.
## Document uploads
`PUT /{resource}/{id}/upload` exists for 16 resources, each getting
@@ -359,10 +391,8 @@ smoke test does not exercise them.
### Nothing is blocked on a dependency
Multipart uploads used to be, and are now wrapped — see [Document
uploads](#document-uploads). One upload route is deliberately left out:
`PUT /edocument/upload` is not a document attachment. It imports an expense
from an e-document (`ExpenseController::edocument`), and is better treated as
an import endpoint alongside the others under *Import/export* below.
uploads](#document-uploads) and [Import and
export](#import-and-export).
### Resource groups not wrapped
@@ -383,8 +413,8 @@ check `routes/api.php`).
`POST /user/{id}/reconfirm`
- **Auth** — `/login`, `/logout`, `/refresh`, `/connected_account`,
passkey routes
- **Import/export** — `/import`, `/preimport`, `/import_json`, `/export`,
`/migration/start`
- **Migration** — `/migration/start`. Moves a company between instances and
is driven by the v4 migration tool, not by an SDK caller.
- **Invoice extras** — `delivery_note`, `payment_schedule`,
`update_reminders`
- **Client-portal invitation downloads** — the `routes/client.php` family:
+1 -1
View File
@@ -100,7 +100,7 @@ IN_API_TOKEN=... cargo run --example list_clients
## Coverage
Every one of the API's 34 resource groups, plus users — 333 methods. Each
Every one of the API's 34 resource groups, plus users — 338 methods. Each
resource has `list`, `list_all`, `get`, `create`, `update`, `delete`, and
(where the API offers one) `bulk`:
+134 -5
View File
@@ -8,11 +8,11 @@ use crate::error::{ApiErrorBody, Error, Result};
use crate::models::{
Activity, ActivityFeedEntry, BankIntegration, BankTransaction, BankTransactionRule,
BulkRequest, Client as ClientModel, ClientGatewayToken, Company, CompanyGateway, CompanyToken,
Credit, Design, Document, Expense, ExpenseCategory, GroupSetting, HealthCheck, Invoice,
ItemResponse, ListParams, ListResponse, Location, MessageResponse, Payment, PaymentTerm,
Product, Project, PurchaseOrder, Quote, RecurringExpense, RecurringInvoice, RecurringQuote,
SearchResults, Statics, Subscription, SubscriptionStep, SystemLog, Tag, Task, TaskScheduler,
TaskStatus, TaxRate, User, Vendor, Webhook,
Credit, Design, Document, Expense, ExpenseCategory, ExportJob, GroupSetting, HealthCheck,
ImportPreview, Invoice, ItemResponse, ListParams, ListResponse, Location, MessageResponse,
Payment, PaymentTerm, Product, Project, PurchaseOrder, Quote, RecurringExpense,
RecurringInvoice, RecurringQuote, SearchResults, Statics, Subscription, SubscriptionStep,
SystemLog, Tag, Task, TaskScheduler, TaskStatus, TaxRate, User, Vendor, Webhook,
};
use crate::retry::{self, RetryPolicy};
use crate::upload::FileUpload;
@@ -629,6 +629,31 @@ impl InvoiceNinjaClient {
Ok(body.data)
}
/// Sends a caller-built multipart form and decodes a body with no `data`
/// wrapper.
///
/// [`upload_files`](Self::upload_files) covers the 16 `/{id}/upload`
/// routes, which all take `documents[]` and answer with an envelope. The
/// import and e-document routes share none of that: the field name is
/// `files`, keyed by entity type for the CSV importers, and the reply is
/// a bare `{"message": …}`. So the form is built at the call site, where
/// the route's own shape is visible, and only the dispatch is shared.
///
/// Always a `POST` on the wire. A route registered as `PUT` needs a
/// `_method=PUT` text field instead, for the reason spelled out on
/// `upload_files`. Like every multipart request, these are never retried:
/// the body is a stream, so `execute` gives it exactly one attempt.
async fn multipart_unwrapped<T: DeserializeOwned>(
&self,
path: &str,
form: multipart::Form,
) -> Result<T> {
let response = self
.execute(self.req(Method::POST, path).multipart(form))
.await?;
Self::decode(response).await
}
/// Issues a request whose body is discarded, failing on non-2xx.
async fn get_discard(&self, path: &str) -> Result<()> {
self.get_raw(path).await.map(|_| ())
@@ -1981,3 +2006,107 @@ impl InvoiceNinjaClient {
self.post_raw("live_preview/purchase_order", body).await
}
}
/// Import, export, and the third-party integrations.
///
/// These are the least SDK-shaped part of the API, and they are typed
/// accordingly. Most return `serde_json::Value` rather than a model: their
/// replies are pass-throughs from an external service (Storecove, Stripe,
/// Yodlee, GoCardless) or ad-hoc arrays assembled for Invoice Ninja's own
/// React client, so they have no stable documented schema. The convention
/// matches `client_peppol_discovery`, which has returned `Value` since it
/// landed. Where a shape *is* stable and is the point of the call, it is
/// typed: [`ExportJob`] and [`ImportPreview`].
///
/// Every write here is a `POST`, `PUT`, or `DELETE`, so none is retried on a
/// 5xx. Several queue background work and answer `{"message": "Processing"}`
/// before anything has happened, so a success here means accepted, not done.
impl InvoiceNinjaClient {
/// Queues a full company export and returns the link to collect it from.
///
/// The work happens in a queued job, so this returns as soon as it is
/// accepted. See [`ExportJob`] for the expiry on the URL.
///
/// The body selects the format and date range (`send_email`, `report_keys`,
/// `start_date`, `end_date`); an empty object exports everything.
pub async fn export<B: Serialize>(&self, body: &B) -> Result<ExportJob> {
self.post_unwrapped("export", body).await
}
/// Uploads CSVs and returns the column mapping the server suggests —
/// the first half of a CSV import.
///
/// Each entry pairs an entity type (`client`, `invoice`, `product`,
/// `payment`, `expense`, `vendor`, …) with that entity's CSV, and goes
/// out as `files[<entity_type>]`. Take the `hash` off the result and pass
/// it to [`import`](Self::import) to start the ingest.
///
/// The server caches the uploaded files against that hash for 20 minutes,
/// so the second call has to follow reasonably promptly.
pub async fn preimport(&self, files: Vec<(String, FileUpload)>) -> Result<ImportPreview> {
if files.is_empty() {
return Err(Error::InvalidUpload("no files to import".into()));
}
let mut form = multipart::Form::new();
for (entity_type, file) in files {
form = form.part(format!("files[{entity_type}]"), file.into_part()?);
}
self.multipart_unwrapped("preimport", form).await
}
/// Starts a CSV import from a [`preimport`](Self::preimport) hash.
///
/// The body carries the `hash` plus the column mapping to use — the
/// shape `preimport` described. With a hash present the server never
/// looks at uploaded files, so this goes out as ordinary JSON.
///
/// Only this two-phase form is wrapped. The route also accepts the files
/// and the mapping together in one multipart request, but a mapping is a
/// nested structure, and bracket-encoding nested structures into form
/// fields is the thing this crate avoids by sending JSON everywhere else.
/// Call `preimport` first instead.
///
/// The ingest is queued, so a success means accepted. Rows that fail to
/// import are reported by email, not here.
pub async fn import<B: Serialize>(&self, body: &B) -> Result<MessageResponse> {
self.post_unwrapped("import", body).await
}
/// Restores a company from a `.zip` backup produced by
/// [`export`](Self::export).
///
/// One file, sent as `files`. The reply's `success` flag and any
/// `metadata` the server attaches arrive in
/// [`MessageResponse::extra`](crate::models::MessageResponse).
///
/// **This overwrites the target company.** The route exists for
/// migrations, not for merging two datasets.
pub async fn import_json(&self, file: FileUpload) -> Result<MessageResponse> {
let form = multipart::Form::new().part("files", file.into_part()?);
self.multipart_unwrapped("import_json", form).await
}
/// Imports expenses from e-documents (an XML invoice, or a PDF with one
/// embedded).
///
/// Not a document attachment, which is why it is not one of the
/// `upload_*` methods: the files are parsed and become *expenses*, and
/// nothing is attached to the record you might expect. The route is
/// registered as `PUT`, so this goes out as a `POST` with `_method=PUT`
/// for the usual multipart reason.
///
/// Parsing is queued per file, so the reply says `Processing....` and a
/// file that fails to parse is not reported here.
pub async fn upload_edocument(&self, files: Vec<FileUpload>) -> Result<MessageResponse> {
if files.is_empty() {
return Err(Error::InvalidUpload("no files to upload".into()));
}
let mut form = multipart::Form::new().text("_method", "PUT");
for file in files {
form = form.part("documents[]", file.into_part()?);
}
self.multipart_unwrapped("edocument/upload", form).await
}
}
+51
View File
@@ -0,0 +1,51 @@
//! The company import and export jobs.
//!
//! Both endpoints queue work rather than doing it, so neither reply carries
//! the result — an export answers with the URL to fetch later, and an import
//! answers with nothing but an acknowledgement (see `MessageResponse`).
use serde::{Deserialize, Serialize};
use serde_json::{Map, Value};
/// The reply to `POST /export`.
///
/// The export itself is queued, so `url` is a promise rather than a ready
/// file: fetching it before the job finishes gets nothing. The link points at
/// the `protected_download` route and expires — in one hour, or five if the
/// company has more than 10,000 activities — so it is worth following
/// promptly rather than storing.
///
/// `url` is the only reason to call this endpoint, which is why it is typed
/// rather than left in `extra`.
#[derive(Debug, Clone, Default, Serialize, Deserialize, PartialEq)]
pub struct ExportJob {
#[serde(default, deserialize_with = "super::common::null_to_default")]
pub message: String,
#[serde(default, deserialize_with = "super::common::null_to_default")]
pub url: String,
#[serde(flatten)]
pub extra: Map<String, Value>,
}
/// The reply to `POST /preimport` — the first half of a CSV import.
///
/// `hash` is the handle to the uploaded files, which the server caches for 20
/// minutes. Pass it back in the `import` body to start the ingest; without it
/// the files would have to be uploaded again.
///
/// `mappings` is keyed by entity type (`client`, `invoice`, …), one entry per
/// file uploaded. Each value describes what that file's columns could map to:
/// `available` lists the importable fields, `headers` holds the first two
/// rows, and `hints` is the server's guess at a column mapping. Those three
/// stay `Value` because the upstream shapes come from each entity's own
/// `importable()` map and are not documented; a wrong concrete type here
/// would be a hard decode error, where `Value` is lossless.
#[derive(Debug, Clone, Default, Serialize, Deserialize, PartialEq)]
pub struct ImportPreview {
#[serde(default, deserialize_with = "super::common::null_to_default")]
pub hash: String,
#[serde(default, deserialize_with = "super::common::null_to_default")]
pub mappings: Map<String, Value>,
#[serde(flatten)]
pub extra: Map<String, Value>,
}
+2
View File
@@ -14,6 +14,7 @@ mod expense;
mod expense_category;
mod group_setting;
mod health;
mod import_export;
mod invoice;
mod location;
mod payment;
@@ -54,6 +55,7 @@ pub use expense::Expense;
pub use expense_category::ExpenseCategory;
pub use group_setting::GroupSetting;
pub use health::{HealthCheck, PhpVersion};
pub use import_export::{ExportJob, ImportPreview};
pub use invoice::{Credit, Invoice, InvoiceItem, Quote};
pub use location::Location;
pub use payment::{Payment, PaymentInvoice};
+171
View File
@@ -2365,3 +2365,174 @@ async fn an_empty_invitation_key_is_refused_before_the_request() {
assert!(matches!(err, Error::InvalidPathSegment(_)), "got {err:?}");
assert!(server.received_requests().await.unwrap().is_empty());
}
// --- import and export --------------------------------------------------
#[tokio::test]
async fn export_posts_and_returns_the_collection_url() {
let server = route_server(
"POST",
"/export",
json!({ "message": "Processing", "url": "https://x.test/protected_download/abc" }),
)
.await;
let job = client(&server)
.export(&json!({ "send_email": false }))
.await
.expect("export should be accepted");
assert_eq!(job.message, "Processing");
assert_eq!(job.url, "https://x.test/protected_download/abc");
}
#[tokio::test]
async fn preimport_sends_each_csv_under_its_entity_type() {
let server = MockServer::start().await;
Mock::given(path("/preimport"))
.respond_with(ResponseTemplate::new(200).set_body_json(json!({
"hash": "h1",
"mappings": {
"client": { "available": { "name": "Name" }, "headers": [["name"]], "hints": [0] }
}
})))
.mount(&server)
.await;
let preview = client(&server)
.preimport(vec![
("client".to_string(), pdf("clients.csv")),
("invoice".to_string(), pdf("invoices.csv")),
])
.await
.expect("preimport should succeed");
assert_eq!(preview.hash, "h1");
assert!(preview.mappings.contains_key("client"));
let sent = &server.received_requests().await.unwrap()[0];
assert_eq!(sent.method.as_str(), "POST");
let body = body_text(sent);
// Keyed by entity type, not `documents[]` — a different field name from
// the `/{id}/upload` routes.
assert!(
body.contains("name=\"files[client]\"; filename=\"clients.csv\""),
"body: {body}"
);
assert!(
body.contains("name=\"files[invoice]\"; filename=\"invoices.csv\""),
"body: {body}"
);
assert!(!body.contains("_method"), "preimport is a real POST");
}
#[tokio::test]
async fn preimport_with_no_files_is_refused_before_the_request() {
let server = MockServer::start().await;
Mock::given(path("/preimport"))
.respond_with(ResponseTemplate::new(200).set_body_json(json!({ "hash": "h1" })))
.mount(&server)
.await;
let err = client(&server)
.preimport(vec![])
.await
.expect_err("an empty import must not be sent");
assert!(matches!(err, Error::InvalidUpload(_)), "got {err:?}");
assert!(server.received_requests().await.unwrap().is_empty());
}
/// The second phase is ordinary JSON, not multipart: the hash stands in for
/// the files, so there is nothing to encode as a form.
#[tokio::test]
async fn import_sends_json_rather_than_a_form() {
let server = route_server("POST", "/import", json!({ "message": "Import started" })).await;
let got = client(&server)
.import(&json!({ "hash": "h1", "column_map": { "client": { "mapping": {} } } }))
.await
.expect("import should be accepted");
assert_eq!(got.message, "Import started");
let sent = &server.received_requests().await.unwrap()[0];
let content_type = sent.headers.get("content-type").unwrap().to_str().unwrap();
assert_eq!(content_type, "application/json");
assert_eq!(
sent.body_json::<serde_json::Value>().unwrap()["hash"],
"h1",
"the hash is what stands in for the upload"
);
}
#[tokio::test]
async fn import_json_sends_one_file_named_files() {
let server = route_server(
"POST",
"/import_json",
json!({ "message": "Processing", "success": true }),
)
.await;
let got = client(&server)
.import_json(pdf("backup.zip"))
.await
.expect("import_json should be accepted");
assert_eq!(got.message, "Processing");
assert_eq!(
got.extra["success"], true,
"the success flag is kept in extra"
);
let body = body_text(&server.received_requests().await.unwrap()[0]);
assert!(
body.contains("name=\"files\"; filename=\"backup.zip\""),
"singular `files`, not `files[]`: body: {body}"
);
}
#[tokio::test]
async fn upload_edocument_is_a_post_that_spoofs_put() {
let server = route_server(
"POST",
"/edocument/upload",
json!({ "message": "Processing...." }),
)
.await;
let got = client(&server)
.upload_edocument(vec![pdf("einvoice.xml")])
.await
.expect("edocument upload should be accepted");
assert_eq!(got.message, "Processing....");
let sent = &server.received_requests().await.unwrap()[0];
assert_eq!(
sent.method.as_str(),
"POST",
"the route is a PUT, but a real PUT drops the files"
);
let body = body_text(sent);
assert!(body.contains("name=\"_method\"\r\n\r\nPUT"), "body: {body}");
assert!(
body.contains("name=\"documents[]\"; filename=\"einvoice.xml\""),
"body: {body}"
);
}
#[tokio::test]
async fn edocument_uploads_are_never_replayed() {
let server = MockServer::start().await;
Mock::given(method("POST"))
.respond_with(ResponseTemplate::new(500))
.mount(&server)
.await;
client(&server)
.with_retry_policy(quick_retry())
.upload_edocument(vec![pdf("einvoice.xml")])
.await
.expect_err("500");
assert_eq!(
server.received_requests().await.unwrap().len(),
1,
"a multipart body is not cloneable, so it gets one attempt"
);
}
+2
View File
@@ -66,7 +66,9 @@ fn every_model_absorbs_a_null_or_missing_field() {
Document,
Expense,
ExpenseCategory,
ExportJob,
GroupSetting,
ImportPreview,
Invoice,
InvoiceItem,
Location,