Skip to content

Latest commit

 

History

History
261 lines (210 loc) · 18.6 KB

INSTALL.md

File metadata and controls

261 lines (210 loc) · 18.6 KB

Install

Unsupported commercetools-labs reference code, provided AS IS. See SUPPORT.md.

How to run Contentools locally and deploy it with commercetools Connect. Allow about an hour for the first deployment. Placeholders are written <like-this>; never paste real secrets into a file that is committed.

The connector has three applications (see connect.yaml): service (the content API, served at /service), mcp (the MCP server, served at /mcp, optional) and admin-app (the Merchant Center custom application). They are deployed together as one Connect connector, into one commercetools project.

1. What you need

  • A commercetools project and Organization Admin rights in the Merchant Center (to register a custom application and to grant team permissions).
  • Node.js 22, and Yarn 1 (service/, admin-app/, mcp/) plus Corepack for Yarn 4 (puck/, corepack enable).
  • The commercetools CLI: npm install -g @commercetools/cli, then log in with an API client that has the deployer scopes below: commercetools auth login --client-credentials ....
  • A public git repository (or fork) that Connect can clone, tagged for each release (for example v1.0.0).
  • Docker, only if you want to run commercetools connect validate or commercetools connect application ... locally.

2. Pre-flight: two sets of scopes

Do not mix them up.

The deploying client (the API client you log in to the CLI with) needs:

Scope Why
manage_connectors Create and publish the connector
manage_connectors_deployments Create, redeploy and delete deployments
manage_api_clients Connect creates the runtime API client below

The runtime client is generated by Connect for each deployment from inheritAs.apiClient in connect.yaml, and shared by every application of the connector:

Scope Why
manage_key_value_documents All content is stored as custom objects
view_products The product picker and the product datasources use Product Search

Product Search must be activated for the project (Merchant Center, Settings, Project settings, Storefront Search) if you want the product picker; everything else works without it.

3. Run locally

Run locally before any registration exists; it catches credential and data problems early.

Service (port 8080). Create an API client in the Merchant Center with exactly the two runtime scopes above, then:

cd service
cp .env.example .env     # CTP_* values of that client, and a SERVICE_API_TOKEN (>= 32 characters)
yarn install --frozen-lockfile
yarn start:dev           # http://localhost:8080/service/health

Generate a token with openssl rand -base64 48. The service-side keys are listed in section 5 and in service/README.md.

Playground (port 5174), a small host app that mounts the managers against the service:

cd puck && yarn install --immutable && yarn build
cp playground/.env.example playground/.env   # VITE_BASE_URL, VITE_BUSINESS_UNIT_KEY, VITE_SERVICE_TOKEN
yarn start:playground                        # http://localhost:5174

VITE_SERVICE_TOKEN is the SERVICE_API_TOKEN of your local service (local development only: Vite bundles it into the browser). More in puck/playground/README.md.

Merchant Center app (port 3001), after the registration of section 4:

cd admin-app
cp .env.example .env     # CLOUD_IDENTIFIER, CTP_INITIAL_PROJECT_KEY, ENTRY_POINT_URI_PATH, CUSTOM_APPLICATION_ID, CONTENTOOLS_SERVICE_URL
yarn install --frozen-lockfile
yarn start               # http://localhost:3001, redirects to the Merchant Center login

CLOUD_IDENTIFIER is the region of the project: gcp-eu, gcp-us, gcp-au, aws-eu or aws-us. CONTENTOOLS_SERVICE_URL must be an https URL ending in /service (use a tunnel for a local service); see admin-app/README.md. While the puck packages are not published to npm, use yarn admin-app:local-puck from the repository root as described there.

MCP server (port 8080 locally, so not at the same time as the service): see mcp/README.md.

4. Register the Merchant Center custom application (before the first deployment)

Register first with a placeholder URL, because Connect only gives you the application URL after deploying and the deployment needs the Application ID. In the Merchant Center, profile menu, Manage organizations & teams, your organization, Custom Applications:

  1. Add and verify a contact e-mail if asked (once per organization).
  2. Add a custom application:
    • Name: Contentools
    • Application URL: https://example.com (placeholder, replaced in section 7)
    • Entry point URI path: contentools (must equal ENTRY_POINT_URI_PATH and be unique per region)
  3. Permissions: the OAuth scopes are the ones in admin-app/custom-application-config.mjs; copy them from the file instead of retyping (sed -n '/oAuthScopes/,/}/p' admin-app/custom-application-config.mjs). Today: View view_business_units, Manage manage_key_value_documents.
  4. Main menu link: label Contentools, enable the View and Manage permissions for it.
  5. Save, then copy the Application ID (needed for CUSTOM_APPLICATION_ID).

Menu labels may change in the Merchant Center; go by the names above.

5. Deploy on Connect, first pass

Configuration is passed with --configuration. Keys of one application are prefixed with its name (service.KEY, mcp.KEY, admin-app.KEY); CLOUD_IDENTIFIER is shared and unprefixed. Pass every key you want kept on every create, preview and redeploy: keys not declared in connect.yaml are dropped, and a value left out of a redeploy may be lost (check with deployment describe).

# validate the manifest, build and tests (needs Docker)
commercetools connect validate .

# make the repository tag known to Connect
commercetools connect connectorstaged create --key contentools --name "Contentools" \
  --repository-url <repository-url> --repository-tag <tag> --creator-email <email> \
  --supported-regions <connect-region> --integration-types other

# first deployment; use the Connect region that matches the project's region
commercetools connect connectorstaged preview --key contentools --deployment-key <deployment-key> \
  --region <connect-region> \
  --configuration CLOUD_IDENTIFIER=<cloud-id> \
  --configuration admin-app.CUSTOM_APPLICATION_ID=<application-id> \
  --configuration admin-app.ENTRY_POINT_URI_PATH=contentools \
  --configuration admin-app.CONTENTOOLS_SERVICE_URL=https://pending.invalid/service \
  --configuration service.SERVICE_API_TOKEN=<token> \
  --configuration mcp.MCP_ACCESS_TOKEN=<mcp-token> \
  --configuration mcp.SERVICE_API_TOKEN=<token> \
  --configuration mcp.CONTENT_SERVICE_URL=<placeholder-https-url>/service

Sandbox and production deployments are created from a published connector with commercetools connect deployment create --connector-key contentools --region <connect-region> --type sandbox --key <deployment-key> --configuration ...; the exact sequence for each deployment type is in service/README.md and admin-app/README.md. Region names: gcp-eu is europe-west1.gcp, gcp-us is us-central1.gcp, gcp-au is australia-southeast1.gcp, aws-eu is eu-central-1.aws, aws-us is us-east-2.aws.

SERVICE_API_TOKEN is secured. Generate it with openssl rand -base64 48 (at least 32 characters) and give the same value to service.SERVICE_API_TOKEN and mcp.SERVICE_API_TOKEN. MCP_ACCESS_TOKEN must be a different value. Skip the mcp.* lines if you do not deploy the MCP application (it is still part of connect.yaml; MCP_ACCESS_TOKEN, SERVICE_API_TOKEN and CONTENT_SERVICE_URL are required keys of that application).

Configuration keys (generated from connect.yaml; the READMEs of each application carry the same keys):

Key Application Secured Required Default Meaning
MAX_VERSIONS service no no 5 Maximum number of saved versions kept per page or content item; older versions are pruned on save.
LOG_LEVEL service no no info Log level: error, warn, info or debug.
CONTAINER_PREFIX service no no - Prefix added to every Custom Object container name; empty for production. Lets tests isolate data (for example contentools-test-). Changing it on a live deployment hides existing content.
CORS_ALLOWED_ORIGINS service no no - Comma-separated exact browser origins allowed to call the service (storefronts). No default: empty allows no browser origin. The Merchant Center hop is server-to-server and needs no CORS.
MC_REQUIRED_PERMISSION service no no - Optional Merchant Center permission a user must have for writes (for example canManageContentools). Empty means any user with access to the project.
PUBLIC_DATASOURCE_RESOLVE service no no false true makes resolving the built-in product-by-sku and products-by-sku datasources public (rate limited). Default false.
PUBLIC_RESOLVE_MAX_PER_MINUTE service no no 120 Global rate limit per minute for public datasource resolve.
MEDIA_STORAGE_PROVIDER service no no none Media storage backend: none, gcp, aws or cloudinary. With none the upload and media-library endpoints answer 501.
MEDIA_PATH_PREFIX service no no contentools/media Object key prefix for uploaded media, without leading or trailing slash.
MEDIA_MAX_UPLOAD_BYTES service no no 10485760 Largest accepted upload in bytes (1024 to 26214400).
MEDIA_ALLOWED_MIME_TYPES service no no image/jpeg,image/png,image/gif,image/webp,image/avif,application/pdf Comma-separated allowed upload content types. Add image/svg+xml only if you accept the risk.
MEDIA_PUBLIC_BASE_URL service no no - Optional https base URL (CDN) that serves the bucket; empty uses the provider public URL.
MEDIA_GCP_BUCKET service no no - Bucket name; required when MEDIA_STORAGE_PROVIDER is gcp.
MEDIA_AWS_BUCKET service no no - Bucket name; required when MEDIA_STORAGE_PROVIDER is aws.
MEDIA_AWS_REGION service no no us-east-1 AWS region of the bucket.
MEDIA_AWS_ENDPOINT service no no - Optional S3-compatible endpoint URL (for example a MinIO server); empty means Amazon S3.
MEDIA_CLOUDINARY_CLOUD_NAME service no no - Cloud name; required when MEDIA_STORAGE_PROVIDER is cloudinary.
SERVICE_API_TOKEN service yes no - Static bearer token (at least 32 characters) for server-to-server callers such as the MCP application. Empty disables token authentication; Merchant Center session tokens still work.
MEDIA_GCP_SERVICE_ACCOUNT_KEY_B64 service yes no - Service-account JSON key, base64 on one line; required when MEDIA_STORAGE_PROVIDER is gcp.
MEDIA_AWS_ACCESS_KEY_ID service yes no - Access key id; required when MEDIA_STORAGE_PROVIDER is aws.
MEDIA_AWS_SECRET_ACCESS_KEY service yes no - Secret access key; required when MEDIA_STORAGE_PROVIDER is aws.
MEDIA_CLOUDINARY_API_KEY service yes no - API key; required when MEDIA_STORAGE_PROVIDER is cloudinary.
MEDIA_CLOUDINARY_API_SECRET service yes no - API secret; required when MEDIA_STORAGE_PROVIDER is cloudinary.
CONTENT_SERVICE_URL mcp no yes - External URL of the contentools service application, ending in /service (https, no trailing slash).
DEFAULT_BUSINESS_UNIT_KEY mcp no no default Business unit used when a tool call omits businessUnitKey.
MCP_ENABLE_WRITES mcp no no false true registers the 20 mutating tools; they act with the full authority of SERVICE_API_TOKEN.
MCP_TOOL_GROUPS mcp no no all all, or comma-separated: pages,contents,templates,configuration,datasources,products,media,content-types.
MCP_ALLOWED_ORIGINS mcp no no - Comma-separated browser origins allowed to call /mcp; empty allows only non-browser clients.
MCP_REQUEST_TIMEOUT_MS mcp no no 15000 Timeout in milliseconds for each call from a tool to the service.
MCP_ACCESS_TOKEN mcp yes yes - Bearer token(s) MCP clients must send; a comma-separated list allows rotation; each at least 32 characters.
SERVICE_API_TOKEN mcp yes yes - Static token the content service accepts (same value as the service application's SERVICE_API_TOKEN); at least 32 characters.
CUSTOM_APPLICATION_ID admin-app no yes - The Application ID the Merchant Center shows after the custom application is registered.
ENTRY_POINT_URI_PATH admin-app no yes contentools The entry point URI path the custom application is registered with (unique per region, 2-64 chars, lowercase alphanumerics and hyphens).
CONTENTOOLS_SERVICE_URL admin-app yes no - https URL of the service application, ending in /service. Unknown on the first deployment: pass a placeholder such as https://pending.invalid/service, then redeploy with the real URL.
CLOUD_IDENTIFIER all (shared) no yes - Merchant Center cloud identifier of the project's region (gcp-eu, gcp-us, gcp-au, aws-eu, aws-us); the admin-app's MC API host and the issuer of the forwarded tokens verified by the service.

Provided by Connect, never configured by you: CTP_CLIENT_ID, CTP_CLIENT_SECRET, CTP_SCOPE, CTP_PROJECT_KEY, CTP_API_URL, CTP_AUTH_URL (the generated runtime client), CONNECT_SERVICE_URL (the external URL of a service application; the service uses its origin as token audience), APPLICATION_URL (the URL of the Merchant Center application), PORT.

6. Second pass: the real URLs

Wait for the deployment to show Deployed and read the URLs:

commercetools connect deployment describe --key <deployment-key>

Connect gives no variable holding a sibling application's URL, so give the applications the service URL and redeploy with all keys again:

commercetools connect deployment redeploy --key <deployment-key> \
  --configuration CLOUD_IDENTIFIER=<cloud-id> \
  --configuration admin-app.CUSTOM_APPLICATION_ID=<application-id> \
  --configuration admin-app.ENTRY_POINT_URI_PATH=contentools \
  --configuration admin-app.CONTENTOOLS_SERVICE_URL=<service-url> \
  --configuration service.SERVICE_API_TOKEN=<token> \
  --configuration mcp.MCP_ACCESS_TOKEN=<mcp-token> \
  --configuration mcp.SERVICE_API_TOKEN=<token> \
  --configuration mcp.CONTENT_SERVICE_URL=<service-url>

<service-url> ends in /service. Add --updateConnector only when the tag changed. Also set service.CORS_ALLOWED_ORIGINS (exact origins) if a storefront in a browser reads published content; the Merchant Center hop is server to server and needs no CORS. Media storage keys (service.MEDIA_*) are optional, see service/README.md. Changes can take a minute, a deployment up to about 15 minutes. A new deployment key means new URLs for every application: repeat sections 6 and 7.

7. Publish the URL, set Ready, install, grant permissions

  1. In the Merchant Center registration, replace the placeholder Application URL with the admin-app URL from deployment describe. (Alternatively npx mc-scripts config:sync in admin-app/, which also pushes scopes and menu links from custom-application-config.mjs; it needs npx mc-scripts login.)
  2. Set the custom application's state to Ready if the Merchant Center asks for it.
  3. Install it for the organization or project (Custom Applications, Install).
  4. Give the team that should use it the View permission, and the Manage permission for editors.

8. Verify

  1. Health: curl -fsS https://<service-url>/health answers 200 with {"status":"UP",...} (<service-url> ends in /service).
  2. Protected routes are closed: curl -i https://<service-url>/default/puck-pages answers 401.
  3. Open Contentools in the Merchant Center, create a page, edit it, publish it.
  4. Read it without credentials: curl -fsS https://<service-url>/default/published/puck-pages/<page-key> (the key is shown in the page URL of the editor) answers the published page.
  5. With the token: curl -fsS -H "Authorization: Bearer <token>" https://<service-url>/default/puck-pages.
  6. MCP, if deployed: curl -s -X POST https://<mcp-url>/mcp -H "Authorization: Bearer <mcp-token>" -H 'Content-Type: application/json' -H 'Accept: application/json, text/event-stream' -d '{"jsonrpc":"2.0","id":1,"method":"tools/list"}' lists the tools (<mcp-url> is the mcp URL from deployment describe).

9. Troubleshooting

Symptom Cause and fix
Blank page or 404 for assets in the Merchant Center The Application URL still is the placeholder, or points at the wrong application: use the admin-app URL from deployment describe
"The service URL is not configured" in the app CONTENTOOLS_SERVICE_URL is empty or not https: redeploy with it (section 6)
401 on every request from the Merchant Center app The token audience does not match: CONNECT_SERVICE_URL must be the https origin the app forwards to (the service accepts aud equal to the origin of CONNECT_SERVICE_URL, and the full-path form), or CLOUD_IDENTIFIER is not the project's region
401 from scripts or the MCP server SERVICE_API_TOKEN is wrong or differs between service and mcp
403 for a Merchant Center user The token belongs to another project, or the user lacks MC_REQUIRED_PERMISSION
Browser CORS error from a storefront The origin is not listed in service.CORS_ALLOWED_ORIGINS (exact origin, no path)
First request after idle is slow Sandbox and preview deployments scale to zero; a cold start takes about 15 seconds
A setting has no effect or disappeared The key is not declared in connect.yaml, or it was left out of the last redeploy: send every key again
postDeploy failed Read commercetools connect deployment logs --key <deployment-key> --application service; the lines name the failed check (token, scopes, custom-objects, product-search, storage)
Deployment still shows old behaviour The code update chain was not completed: push the new git tag, point the staged connector at it (connectorstaged update), publish, then deployment redeploy --updateConnector
Build log: Invalid application URL: "" (admin-app) Connect did not provide APPLICATION_URL: see the troubleshooting table of admin-app/README.md
Media endpoints answer 501 MEDIA_STORAGE_PROVIDER is none: set a provider and its keys, then redeploy

More, including day-2 operations: docs/runbook.md.