Dataverse + Cloudflare Integration Playbook

Author: Forrest Zhang

A privacy-safe reference architecture for secure, scalable self-service portals backed by Microsoft Dataverse—written as a reusable technical reference without any product or client branding.

Why this pattern matters

Many organizations that manage sensitive workflows (for example, housing-related services, public-sector intake, regulated operations, or compliance-heavy programs) face the same architectural tension:

  • They need a modern, low-friction portal experience for both anonymous visitors and authenticated users.
  • They must keep authoritative records and permissions inside a governed system of record (Dataverse).
  • They need to support mixed content: public media (images/floor plans) and private documents (legal/internal files).
  • They must prevent browsers from directly calling Dataverse, reducing exposure and simplifying security governance.

This playbook addresses those constraints by using a clean separation of concerns: edge UX + centralized orchestration + governed data. It is intentionally privacy-safe and focuses on the integration approach rather than any specific domain branding.

Core design rule: Never let the portal call Dataverse directly. All portal reads/writes go through BFF endpoints.

1. Purpose

This document summarizes the end-to-end integration pattern used to connect:

  • Dataverse (system of record)
  • Model-driven app (operations UI)
  • Cloudflare-hosted portal (public + authenticated UX)
  • Azure Functions BFF (security and orchestration layer)
  • Cloudflare R2 (public media)
  • SharePoint (private/internal documents)

It is written as a reusable technical reference without product/client branding.

2. Architecture Pattern

2.1 Core principle

Never let the portal call Dataverse directly. All portal reads/writes go through BFF endpoints.

2.2 Runtime components

  • Dataverse: source of truth for business entities and metadata.
  • Model-driven app: internal staff operations.
  • Cloudflare Pages/Workers: portal UI + edge caching + function proxy.
  • Azure Functions (Node.js): BFF, auth checks, Dataverse API calls, upload orchestration.
  • Cloudflare R2: public listing media (images/floor plans).
  • SharePoint: private/internal files and legal documents.

3. Data and File Strategy

3.1 Structured data

Dataverse tables hold authoritative business data (property, unit, inquiry, application, lease, charge, payment, maintenance, etc.).

3.2 Public media

  • Stored in R2.
  • Accessed via CDN/public URL.
  • Dataverse hx_files stores metadata (hx_cdnpath, hx_publicurl, category, visibility, sort order, relation to property/unit).

3.3 Private documents

  • Stored in SharePoint document libraries.
  • Dataverse hx_files stores metadata (hx_sharepointurl, category, relation, visibility flags).
  • Download/read always controlled through secure backend policy.

4. BFF Endpoint Design

4.1 Public endpoints (anonymous, cacheable)

GET /api/public/listings
GET /api/public/listings/{id}
GET /api/public/properties/{id}
POST /api/public/inquiries
POST /api/public/applications

4.2 Secure uploader endpoints (internal key protected)

POST /api/secure/files/upload-session
POST /api/secure/files/register-file

4.3 Backward-compatible legacy endpoints

POST /api/secure/r2/upload-url
POST /api/secure/r2/register-file

4.4 Upload flow (current)

  1. Web resource sends file metadata to /secure/files/upload-session.
  2. BFF determines destination by fileCategory:
    • Photo, Floor Plan -> R2
    • Document, Template, Notice, Other -> SharePoint
  3. BFF returns presigned/upload-session URL.
  4. Browser uploads file directly to destination.
  5. Web resource calls /secure/files/register-file to persist hx_files metadata in Dataverse.

5. Dataverse Form Uploader (Web Resource)

5.1 Resources

hx_/file/script/uploader.html
hx_/file/script/uploader.js
hx_/file/script/uploader.css

5.2 Form configuration

  • Add HTML web resource to target form.
  • Enable pass-through record context parameters.

Custom parameters example:

bffBaseUrl=https://<function-app-host>;apiKeyEnv=hx_R2InternalApiKey;defaultCategory=photo

5.3 UX behaviors implemented

  • Destination hint shown by category.
  • Portal Visible shown for public media flow only.
  • Primary shown for photo only.
  • Successful uploads removed from queue (prevents duplicate upload on next click).
  • Failed items kept for retry.
  • Debug log panel included for support.

6. Azure Function App Configuration

6.1 Core app settings

DATAVERSE_URL
AAD_TENANT_ID
AAD_CLIENT_ID
AAD_CLIENT_SECRET
INTERNAL_API_KEY

6.2 R2 settings

R2_ACCOUNT_ID
R2_ACCESS_KEY_ID
R2_SECRET_ACCESS_KEY
R2_BUCKET
R2_PUBLIC_BASE_URL
R2_ENDPOINT (optional)

6.3 SharePoint settings

SHAREPOINT_SITE_ID
SHAREPOINT_ROOT_FOLDER
"/" means no extra root folder.
SHAREPOINT_DRIVE_ID (fallback/default library)
SHAREPOINT_DRIVE_ID_HX_<ENTITY> (per-table override), for example:
SHAREPOINT_DRIVE_ID_HX_PROPERTY
SHAREPOINT_DRIVE_ID_HX_CHARGE
etc.

7. Microsoft Graph / Entra Requirements

7.1 Required permission model

Use Microsoft Graph -> Application permissions:

Sites.ReadWrite.All

7.2 Critical detail

Token acquisition can succeed while SharePoint calls still fail if app-role assignment/consent is missing.

Verification checklist:

  • App role exists in app registration permissions.
  • Admin consent granted.
  • Client credential token contains roles: ["Sites.ReadWrite.All"].

8. SharePoint Path Logic

8.1 Path behavior

For per-entity drive override, upload path is kept clean:

/<recordId>/<timestamp-file> (when SHAREPOINT_ROOT_FOLDER=/)

For shared/default drive:

/<entityName>/<recordId>/<timestamp-file>

8.2 Why this matters

  • Avoids duplicated segments when library name already equals entity name (example: hx_property).
  • Keeps per-table libraries tidy and predictable.

9. Deployment and Operations

9.1 Build/deploy

Build: npm run build
Publish: func azure functionapp publish <function-app-name> --typescript

9.2 Config updates

  • Use Azure Portal or az functionapp config appsettings set.
  • Restart function app after permission/config changes to refresh cached tokens.

9.3 Validation sequence

  1. Health check: /api/health
  2. Test public listing endpoints.
  3. Test uploader:
    • Photo -> R2 expected
    • Document -> SharePoint expected
  4. Confirm hx_files record created in Dataverse.
  5. Confirm portal displays public media correctly.

10. Common Failure Modes and Fixes

10.1 Missing DATAVERSE_URL
Cause: missing app setting.
Fix: set required env vars in Function App and restart.

10.2 SharePoint General exception while processing
Typical cause: Graph app-role/consent not actually effective.
Fix: validate Sites.ReadWrite.All app-role assignment and token roles; restart Function App.

10.3 Dataverse undeclared property in file registration
Cause: lookup bind property mismatch in payload.
Fix: use robust lookup fallback candidates and register against correct relation.

10.4 Portal not showing new media
Check hx_filecategory, hx_portalvisible, hx_ispublic, relation (property/unit), and listing endpoint mapping/filter logic.

11. Security Model Summary

  • Public endpoints are read/write-limited by route and validation.
  • Secure uploader endpoints require INTERNAL_API_KEY (x-rs-internal-key).
  • Browser uploads directly to storage using short-lived upload URLs.
  • Sensitive/private files routed to SharePoint, public listing media to R2.

12. Suggested Next Enhancements

  • Replace many SHAREPOINT_DRIVE_ID_HX_* env vars with Dataverse-driven mapping table.
  • Add centralized telemetry table/log level strategy for plugins + JS + BFF correlation IDs.
  • Add automated smoke tests for upload-session and register-file endpoints.
  • Add explicit portal rendering rules for floor plan PDF vs image assets.
at

No comments:

Post a Comment

Subscribe to: Post Comments (Atom)