diff --git a/CHANGELOG.md b/CHANGELOG.md index 170cb3d..05b45f5 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,18 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Added +- Import and export, five methods: `export`, `preimport`, `import`, + `import_json`, and `upload_edocument`. A CSV import is two calls — + `preimport` uploads the files as `files[]` and returns the + column mapping the server suggests, then `import` starts the ingest from + the returned `hash`. The route's one-shot form (files and mapping in a + single multipart request) is deliberately not wrapped: a mapping is nested, + and bracket-encoding nested structures is what this crate avoids by sending + JSON. `upload_edocument` imports expenses from e-documents and is not a + document attachment, which is why it is not an `upload_*` method. Adds + `ExportJob` (the export's collection URL expires in 1–5 hours) and + `ImportPreview`. All of these queue work, so a success means accepted, not + done. - Invitation-key downloads, eight methods: `download_invoice_by_invitation_key` and `download_e_invoice_by_invitation_key`, plus the same pair for quotes, diff --git a/CLAUDE.md b/CLAUDE.md index cc4cb1a..926da61 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -337,9 +337,9 @@ changes, doc tweaks) does not need an entry. ## Coverage **All 34 of the API's `Route::resource` groups are wrapped**, plus `users` -(which isn't one). 333 methods: 30 resources with CRUD + bulk, 3 with CRUD +(which isn't one). 338 methods: 30 resources with CRUD + bulk, 3 with CRUD only, 4 with entity actions, 4 with invitation-key downloads, 16 with -document uploads, and 69 hand-written. +document uploads, and 74 hand-written. Plus the non-CRUD surface: documents, entity actions and PDF downloads for the four action-capable resources, their invitation-key PDF and e-document diff --git a/COVERAGE.md b/COVERAGE.md index aacd20b..78afae3 100644 --- a/COVERAGE.md +++ b/COVERAGE.md @@ -18,7 +18,7 @@ declares 34 `Route::resource` groups and 274 explicit route declarations > exists or what it returns. **All 34 `Route::resource` groups are now wrapped**, plus `users` (which is -not a resource group). 333 methods in total. +not a resource group). 338 methods in total. ## Resources with full CRUD + bulk @@ -168,6 +168,38 @@ controllers return that directly rather than raising for model` text a missing record produces elsewhere, but the status is the same 400. +## Import and export + +| Route | Method | Returns | +| --- | --- | --- | +| `POST /export` | `export` | `ExportJob` | +| `POST /preimport` | `preimport` | `ImportPreview` | +| `POST /import` | `import` | `MessageResponse` | +| `POST /import_json` | `import_json` | `MessageResponse` | +| `PUT /edocument/upload` | `upload_edocument` | `MessageResponse` | + +A CSV import is two calls. `preimport` uploads the files — one per entity +type, as `files[]` — and answers with a `hash` plus the column +mapping the server suggests; `import` then starts the ingest from that hash. +The server holds the files against the hash for 20 minutes. + +The route also accepts the files and the mapping together in one multipart +request. That form is deliberately not wrapped: a mapping is a nested +structure, and bracket-encoding nested structures into form fields is exactly +what this crate avoids by sending JSON (see [the PHP SDK +comparison](#compared-with-the-official-php-sdk)). With a `hash` present the +controller never looks at uploaded files, so `import` is plain JSON. + +`upload_edocument` is *not* a document attachment, which is why it is not one +of the `upload_*` methods: `ExpenseController::edocument` parses each file and +creates **expenses**. Like the `/{id}/upload` routes it is registered as `PUT` +and so goes out as a `POST` with `_method=PUT`; unlike them the reply has no +`data` envelope. + +All five queue background work, so a 200 means accepted, not finished. Rows +that fail to import are reported by email. `import_json` restores a company +from an `export` backup and **overwrites the target company**. + ## Document uploads `PUT /{resource}/{id}/upload` exists for 16 resources, each getting @@ -359,10 +391,8 @@ smoke test does not exercise them. ### Nothing is blocked on a dependency Multipart uploads used to be, and are now wrapped — see [Document -uploads](#document-uploads). One upload route is deliberately left out: -`PUT /edocument/upload` is not a document attachment. It imports an expense -from an e-document (`ExpenseController::edocument`), and is better treated as -an import endpoint alongside the others under *Import/export* below. +uploads](#document-uploads) and [Import and +export](#import-and-export). ### Resource groups not wrapped @@ -383,8 +413,8 @@ check `routes/api.php`). `POST /user/{id}/reconfirm` - **Auth** — `/login`, `/logout`, `/refresh`, `/connected_account`, passkey routes -- **Import/export** — `/import`, `/preimport`, `/import_json`, `/export`, - `/migration/start` +- **Migration** — `/migration/start`. Moves a company between instances and + is driven by the v4 migration tool, not by an SDK caller. - **Invoice extras** — `delivery_note`, `payment_schedule`, `update_reminders` - **Client-portal invitation downloads** — the `routes/client.php` family: diff --git a/README.md b/README.md index fe5295f..89f9780 100644 --- a/README.md +++ b/README.md @@ -100,7 +100,7 @@ IN_API_TOKEN=... cargo run --example list_clients ## Coverage -Every one of the API's 34 resource groups, plus users — 333 methods. Each +Every one of the API's 34 resource groups, plus users — 338 methods. Each resource has `list`, `list_all`, `get`, `create`, `update`, `delete`, and (where the API offers one) `bulk`: diff --git a/src/client.rs b/src/client.rs index 86346e1..69cf4a7 100644 --- a/src/client.rs +++ b/src/client.rs @@ -8,11 +8,11 @@ use crate::error::{ApiErrorBody, Error, Result}; use crate::models::{ Activity, ActivityFeedEntry, BankIntegration, BankTransaction, BankTransactionRule, BulkRequest, Client as ClientModel, ClientGatewayToken, Company, CompanyGateway, CompanyToken, - Credit, Design, Document, Expense, ExpenseCategory, GroupSetting, HealthCheck, Invoice, - ItemResponse, ListParams, ListResponse, Location, MessageResponse, Payment, PaymentTerm, - Product, Project, PurchaseOrder, Quote, RecurringExpense, RecurringInvoice, RecurringQuote, - SearchResults, Statics, Subscription, SubscriptionStep, SystemLog, Tag, Task, TaskScheduler, - TaskStatus, TaxRate, User, Vendor, Webhook, + Credit, Design, Document, Expense, ExpenseCategory, ExportJob, GroupSetting, HealthCheck, + ImportPreview, Invoice, ItemResponse, ListParams, ListResponse, Location, MessageResponse, + Payment, PaymentTerm, Product, Project, PurchaseOrder, Quote, RecurringExpense, + RecurringInvoice, RecurringQuote, SearchResults, Statics, Subscription, SubscriptionStep, + SystemLog, Tag, Task, TaskScheduler, TaskStatus, TaxRate, User, Vendor, Webhook, }; use crate::retry::{self, RetryPolicy}; use crate::upload::FileUpload; @@ -629,6 +629,31 @@ impl InvoiceNinjaClient { Ok(body.data) } + /// Sends a caller-built multipart form and decodes a body with no `data` + /// wrapper. + /// + /// [`upload_files`](Self::upload_files) covers the 16 `/{id}/upload` + /// routes, which all take `documents[]` and answer with an envelope. The + /// import and e-document routes share none of that: the field name is + /// `files`, keyed by entity type for the CSV importers, and the reply is + /// a bare `{"message": …}`. So the form is built at the call site, where + /// the route's own shape is visible, and only the dispatch is shared. + /// + /// Always a `POST` on the wire. A route registered as `PUT` needs a + /// `_method=PUT` text field instead, for the reason spelled out on + /// `upload_files`. Like every multipart request, these are never retried: + /// the body is a stream, so `execute` gives it exactly one attempt. + async fn multipart_unwrapped( + &self, + path: &str, + form: multipart::Form, + ) -> Result { + let response = self + .execute(self.req(Method::POST, path).multipart(form)) + .await?; + Self::decode(response).await + } + /// Issues a request whose body is discarded, failing on non-2xx. async fn get_discard(&self, path: &str) -> Result<()> { self.get_raw(path).await.map(|_| ()) @@ -1981,3 +2006,107 @@ impl InvoiceNinjaClient { self.post_raw("live_preview/purchase_order", body).await } } + +/// Import, export, and the third-party integrations. +/// +/// These are the least SDK-shaped part of the API, and they are typed +/// accordingly. Most return `serde_json::Value` rather than a model: their +/// replies are pass-throughs from an external service (Storecove, Stripe, +/// Yodlee, GoCardless) or ad-hoc arrays assembled for Invoice Ninja's own +/// React client, so they have no stable documented schema. The convention +/// matches `client_peppol_discovery`, which has returned `Value` since it +/// landed. Where a shape *is* stable and is the point of the call, it is +/// typed: [`ExportJob`] and [`ImportPreview`]. +/// +/// Every write here is a `POST`, `PUT`, or `DELETE`, so none is retried on a +/// 5xx. Several queue background work and answer `{"message": "Processing"}` +/// before anything has happened, so a success here means accepted, not done. +impl InvoiceNinjaClient { + /// Queues a full company export and returns the link to collect it from. + /// + /// The work happens in a queued job, so this returns as soon as it is + /// accepted. See [`ExportJob`] for the expiry on the URL. + /// + /// The body selects the format and date range (`send_email`, `report_keys`, + /// `start_date`, `end_date`); an empty object exports everything. + pub async fn export(&self, body: &B) -> Result { + self.post_unwrapped("export", body).await + } + + /// Uploads CSVs and returns the column mapping the server suggests — + /// the first half of a CSV import. + /// + /// Each entry pairs an entity type (`client`, `invoice`, `product`, + /// `payment`, `expense`, `vendor`, …) with that entity's CSV, and goes + /// out as `files[]`. Take the `hash` off the result and pass + /// it to [`import`](Self::import) to start the ingest. + /// + /// The server caches the uploaded files against that hash for 20 minutes, + /// so the second call has to follow reasonably promptly. + pub async fn preimport(&self, files: Vec<(String, FileUpload)>) -> Result { + if files.is_empty() { + return Err(Error::InvalidUpload("no files to import".into())); + } + + let mut form = multipart::Form::new(); + for (entity_type, file) in files { + form = form.part(format!("files[{entity_type}]"), file.into_part()?); + } + self.multipart_unwrapped("preimport", form).await + } + + /// Starts a CSV import from a [`preimport`](Self::preimport) hash. + /// + /// The body carries the `hash` plus the column mapping to use — the + /// shape `preimport` described. With a hash present the server never + /// looks at uploaded files, so this goes out as ordinary JSON. + /// + /// Only this two-phase form is wrapped. The route also accepts the files + /// and the mapping together in one multipart request, but a mapping is a + /// nested structure, and bracket-encoding nested structures into form + /// fields is the thing this crate avoids by sending JSON everywhere else. + /// Call `preimport` first instead. + /// + /// The ingest is queued, so a success means accepted. Rows that fail to + /// import are reported by email, not here. + pub async fn import(&self, body: &B) -> Result { + self.post_unwrapped("import", body).await + } + + /// Restores a company from a `.zip` backup produced by + /// [`export`](Self::export). + /// + /// One file, sent as `files`. The reply's `success` flag and any + /// `metadata` the server attaches arrive in + /// [`MessageResponse::extra`](crate::models::MessageResponse). + /// + /// **This overwrites the target company.** The route exists for + /// migrations, not for merging two datasets. + pub async fn import_json(&self, file: FileUpload) -> Result { + let form = multipart::Form::new().part("files", file.into_part()?); + self.multipart_unwrapped("import_json", form).await + } + + /// Imports expenses from e-documents (an XML invoice, or a PDF with one + /// embedded). + /// + /// Not a document attachment, which is why it is not one of the + /// `upload_*` methods: the files are parsed and become *expenses*, and + /// nothing is attached to the record you might expect. The route is + /// registered as `PUT`, so this goes out as a `POST` with `_method=PUT` + /// for the usual multipart reason. + /// + /// Parsing is queued per file, so the reply says `Processing....` and a + /// file that fails to parse is not reported here. + pub async fn upload_edocument(&self, files: Vec) -> Result { + if files.is_empty() { + return Err(Error::InvalidUpload("no files to upload".into())); + } + + let mut form = multipart::Form::new().text("_method", "PUT"); + for file in files { + form = form.part("documents[]", file.into_part()?); + } + self.multipart_unwrapped("edocument/upload", form).await + } +} diff --git a/src/models/import_export.rs b/src/models/import_export.rs new file mode 100644 index 0000000..2c278bd --- /dev/null +++ b/src/models/import_export.rs @@ -0,0 +1,51 @@ +//! The company import and export jobs. +//! +//! Both endpoints queue work rather than doing it, so neither reply carries +//! the result — an export answers with the URL to fetch later, and an import +//! answers with nothing but an acknowledgement (see `MessageResponse`). + +use serde::{Deserialize, Serialize}; +use serde_json::{Map, Value}; + +/// The reply to `POST /export`. +/// +/// The export itself is queued, so `url` is a promise rather than a ready +/// file: fetching it before the job finishes gets nothing. The link points at +/// the `protected_download` route and expires — in one hour, or five if the +/// company has more than 10,000 activities — so it is worth following +/// promptly rather than storing. +/// +/// `url` is the only reason to call this endpoint, which is why it is typed +/// rather than left in `extra`. +#[derive(Debug, Clone, Default, Serialize, Deserialize, PartialEq)] +pub struct ExportJob { + #[serde(default, deserialize_with = "super::common::null_to_default")] + pub message: String, + #[serde(default, deserialize_with = "super::common::null_to_default")] + pub url: String, + #[serde(flatten)] + pub extra: Map, +} + +/// The reply to `POST /preimport` — the first half of a CSV import. +/// +/// `hash` is the handle to the uploaded files, which the server caches for 20 +/// minutes. Pass it back in the `import` body to start the ingest; without it +/// the files would have to be uploaded again. +/// +/// `mappings` is keyed by entity type (`client`, `invoice`, …), one entry per +/// file uploaded. Each value describes what that file's columns could map to: +/// `available` lists the importable fields, `headers` holds the first two +/// rows, and `hints` is the server's guess at a column mapping. Those three +/// stay `Value` because the upstream shapes come from each entity's own +/// `importable()` map and are not documented; a wrong concrete type here +/// would be a hard decode error, where `Value` is lossless. +#[derive(Debug, Clone, Default, Serialize, Deserialize, PartialEq)] +pub struct ImportPreview { + #[serde(default, deserialize_with = "super::common::null_to_default")] + pub hash: String, + #[serde(default, deserialize_with = "super::common::null_to_default")] + pub mappings: Map, + #[serde(flatten)] + pub extra: Map, +} diff --git a/src/models/mod.rs b/src/models/mod.rs index 488ca46..06cada6 100644 --- a/src/models/mod.rs +++ b/src/models/mod.rs @@ -14,6 +14,7 @@ mod expense; mod expense_category; mod group_setting; mod health; +mod import_export; mod invoice; mod location; mod payment; @@ -54,6 +55,7 @@ pub use expense::Expense; pub use expense_category::ExpenseCategory; pub use group_setting::GroupSetting; pub use health::{HealthCheck, PhpVersion}; +pub use import_export::{ExportJob, ImportPreview}; pub use invoice::{Credit, Invoice, InvoiceItem, Quote}; pub use location::Location; pub use payment::{Payment, PaymentInvoice}; diff --git a/tests/client_requests.rs b/tests/client_requests.rs index 829a88b..1ffaca3 100644 --- a/tests/client_requests.rs +++ b/tests/client_requests.rs @@ -2365,3 +2365,174 @@ async fn an_empty_invitation_key_is_refused_before_the_request() { assert!(matches!(err, Error::InvalidPathSegment(_)), "got {err:?}"); assert!(server.received_requests().await.unwrap().is_empty()); } + +// --- import and export -------------------------------------------------- + +#[tokio::test] +async fn export_posts_and_returns_the_collection_url() { + let server = route_server( + "POST", + "/export", + json!({ "message": "Processing", "url": "https://x.test/protected_download/abc" }), + ) + .await; + + let job = client(&server) + .export(&json!({ "send_email": false })) + .await + .expect("export should be accepted"); + assert_eq!(job.message, "Processing"); + assert_eq!(job.url, "https://x.test/protected_download/abc"); +} + +#[tokio::test] +async fn preimport_sends_each_csv_under_its_entity_type() { + let server = MockServer::start().await; + Mock::given(path("/preimport")) + .respond_with(ResponseTemplate::new(200).set_body_json(json!({ + "hash": "h1", + "mappings": { + "client": { "available": { "name": "Name" }, "headers": [["name"]], "hints": [0] } + } + }))) + .mount(&server) + .await; + + let preview = client(&server) + .preimport(vec![ + ("client".to_string(), pdf("clients.csv")), + ("invoice".to_string(), pdf("invoices.csv")), + ]) + .await + .expect("preimport should succeed"); + assert_eq!(preview.hash, "h1"); + assert!(preview.mappings.contains_key("client")); + + let sent = &server.received_requests().await.unwrap()[0]; + assert_eq!(sent.method.as_str(), "POST"); + let body = body_text(sent); + // Keyed by entity type, not `documents[]` — a different field name from + // the `/{id}/upload` routes. + assert!( + body.contains("name=\"files[client]\"; filename=\"clients.csv\""), + "body: {body}" + ); + assert!( + body.contains("name=\"files[invoice]\"; filename=\"invoices.csv\""), + "body: {body}" + ); + assert!(!body.contains("_method"), "preimport is a real POST"); +} + +#[tokio::test] +async fn preimport_with_no_files_is_refused_before_the_request() { + let server = MockServer::start().await; + Mock::given(path("/preimport")) + .respond_with(ResponseTemplate::new(200).set_body_json(json!({ "hash": "h1" }))) + .mount(&server) + .await; + + let err = client(&server) + .preimport(vec![]) + .await + .expect_err("an empty import must not be sent"); + assert!(matches!(err, Error::InvalidUpload(_)), "got {err:?}"); + assert!(server.received_requests().await.unwrap().is_empty()); +} + +/// The second phase is ordinary JSON, not multipart: the hash stands in for +/// the files, so there is nothing to encode as a form. +#[tokio::test] +async fn import_sends_json_rather_than_a_form() { + let server = route_server("POST", "/import", json!({ "message": "Import started" })).await; + + let got = client(&server) + .import(&json!({ "hash": "h1", "column_map": { "client": { "mapping": {} } } })) + .await + .expect("import should be accepted"); + assert_eq!(got.message, "Import started"); + + let sent = &server.received_requests().await.unwrap()[0]; + let content_type = sent.headers.get("content-type").unwrap().to_str().unwrap(); + assert_eq!(content_type, "application/json"); + assert_eq!( + sent.body_json::().unwrap()["hash"], + "h1", + "the hash is what stands in for the upload" + ); +} + +#[tokio::test] +async fn import_json_sends_one_file_named_files() { + let server = route_server( + "POST", + "/import_json", + json!({ "message": "Processing", "success": true }), + ) + .await; + + let got = client(&server) + .import_json(pdf("backup.zip")) + .await + .expect("import_json should be accepted"); + assert_eq!(got.message, "Processing"); + assert_eq!( + got.extra["success"], true, + "the success flag is kept in extra" + ); + + let body = body_text(&server.received_requests().await.unwrap()[0]); + assert!( + body.contains("name=\"files\"; filename=\"backup.zip\""), + "singular `files`, not `files[]`: body: {body}" + ); +} + +#[tokio::test] +async fn upload_edocument_is_a_post_that_spoofs_put() { + let server = route_server( + "POST", + "/edocument/upload", + json!({ "message": "Processing...." }), + ) + .await; + + let got = client(&server) + .upload_edocument(vec![pdf("einvoice.xml")]) + .await + .expect("edocument upload should be accepted"); + assert_eq!(got.message, "Processing...."); + + let sent = &server.received_requests().await.unwrap()[0]; + assert_eq!( + sent.method.as_str(), + "POST", + "the route is a PUT, but a real PUT drops the files" + ); + let body = body_text(sent); + assert!(body.contains("name=\"_method\"\r\n\r\nPUT"), "body: {body}"); + assert!( + body.contains("name=\"documents[]\"; filename=\"einvoice.xml\""), + "body: {body}" + ); +} + +#[tokio::test] +async fn edocument_uploads_are_never_replayed() { + let server = MockServer::start().await; + Mock::given(method("POST")) + .respond_with(ResponseTemplate::new(500)) + .mount(&server) + .await; + + client(&server) + .with_retry_policy(quick_retry()) + .upload_edocument(vec![pdf("einvoice.xml")]) + .await + .expect_err("500"); + assert_eq!( + server.received_requests().await.unwrap().len(), + 1, + "a multipart body is not cloneable, so it gets one attempt" + ); +} diff --git a/tests/decode_tolerance.rs b/tests/decode_tolerance.rs index 6a087d5..b343bc3 100644 --- a/tests/decode_tolerance.rs +++ b/tests/decode_tolerance.rs @@ -66,7 +66,9 @@ fn every_model_absorbs_a_null_or_missing_field() { Document, Expense, ExpenseCategory, + ExportJob, GroupSetting, + ImportPreview, Invoice, InvoiceItem, Location,