Skip to main content

Fetch Package (@schwab/fetch)

Overview​

The @schwab/fetch package serves as a centralized library of fetch operations and API integrations for the Charles Schwab monorepo. It provides standardized interfaces for communicating with external services including Drupal CMS, Bynder Digital Asset Management, CXP (Customer Experience Platform), Vercel Edge Config, and various third-party APIs.

Architecture​

Package Configuration​

Core Dependencies​

Key Dependencies
{
  "@schwab/schema": "workspace:*",        // Type definitions and validation
  "@schwab/transformer": "workspace:*",   // Data transformation utilities
  "@schwab/utilities": "workspace:*",     // Shared utility functions
  "react": "^19.2.0",                    // React cache integration
  "zod": "^3.24.1"                       // Runtime type validation
}

Module Structure​

Import Map Configuration
{
  "#drupalFetch": "./src/drupal/_drupalFetch.ts",
  "#drupal/*": "./src/drupal/*.ts",
  "#bynderFetch": "./src/bynder/_bynderFetch.ts", 
  "#bynder/*": "./src/bynder/*.ts",
  "#cxpFetch": "./src/cxp/_cxpFetch.ts",
  "#cxp/*": "./src/cxp/*.ts",
  "#utilities/*": "./src/utilities/*.ts",
  "#types": "./src/types/index.ts"
}

Core Service Integrations​

1. Drupal CMS Integration​

Location: src/drupal/

Comprehensive Drupal API integration with content management capabilities:

Drupal Asset Fetching
// src/drupal/cmap-api/getDrupalAsset.ts
import { FetchedDrupalAssetSchema } from '@schwab/schema/fetched/cmap-api/FetchedDrupalAssetSchema';
import { EDrupalSource } from '@schwab/schema/native-enums/EDrupalSource';
import { transformDrupalAsset } from '@schwab/transformer/transformDrupalAsset';

async function fetchAsset(
  nodeId: string,
  datasource: EDrupalSource = EDrupalSource.Education,
  role: ERole,
): Promise<TResponse<TDrupalAsset | null>> {
  const dataSourceUrl: string = getDataSourceUrl(datasource);
  const { status } = await getModerationState();
  const cacheTags = [`${datasource}_node:${nodeId}`];

  const fetchedResponse = await drupalFetch<TFetchedDrupalAsset[]>(
    `${dataSourceUrl}/api/cmap/asset/${status}/en/${nodeId}`,
    datasource,
    cacheTags,
    true,
    true,
  );

  if (!fetchedResponse.isOk) {
    return responseCreate(false, null, fetchedResponse.error);
  }

  // Validate with Zod schema
  const schema = z.array(FetchedDrupalAssetSchema).nonempty();
  const zodResponse = schema.safeParse(fetchedResponse.data);
  
  return zodResponse.success 
    ? responseCreate(true, transformDrupalAsset(zodResponse.data[0], role))
    : responseCreate(false, null, zodResponse.error);
}

Drupal API Functions:

  • getDrupalAsset - Fetch individual content assets
  • getRelatedContent - Content relationship queries
  • getPersonRelatedContent - Person-specific content
  • getTopic - Topic content retrieval
  • getCollection - Content collection management
  • getRedirectsFromDrupal - URL redirect management
  • getTaxonomyList - Taxonomy and categorization

2. Bynder Digital Asset Management​

Location: src/bynder/

Integration with Bynder DAM for digital asset management:

Bynder Integration Examples
// Branch and consultant image fetching
export { getBranchImage } from './src/bynder/getBranchImage.ts';
export { getConsultantImage } from './src/bynder/getConsultantImage.ts';

Features:

  • Branch location image retrieval
  • Consultant profile images
  • Digital asset metadata management
  • Brand asset compliance

3. CXP (Customer Experience Platform)​

Location: src/cxp/

Customer experience and authentication services:

CXP Services
// Authentication and navigation
export { keepAlive } from './src/cxp/keepAlive.ts';
export { getCxpNavigationData } from './src/cxp/getCxpNavigationData.ts';
export { isRequestAuthenticated } from './src/cxp/isRequestAuthenticated.ts';
export { getAltoDataContext } from './src/cxp/getAltoDataContext.ts';

Capabilities:

  • User session management
  • Authentication validation
  • Navigation data retrieval
  • Alto platform integration

4. Vercel Edge Config Integration​

Location: src/edge-config-db/

High-performance edge configuration management:

Edge Config Operations
// Redirect management
export { postRedirectsToEdge } from './src/edge-config-db/postRedirectsToEdge.ts';
export { getRedirectFromEdge } from './src/edge-config-db/getRedirectFromEdge.ts';
export { getAllRedirectsFromEdge } from './src/edge-config-db/getAllRedirectsFromEdge.ts';
export { syncRedirectsToEdge } from './src/edge-config-db/syncRedirectsToEdge.ts';

// Feature flag management
export { getFeatureFlagFromEdge } from './src/edge-config-db/getFeatureFlagFromEdge.ts';
export { getAllFlagsFromEdge } from './src/edge-config-db/getAllFlagsFromEdge.ts';

Advanced Features​

1. Cache Management​

React Cache Integration:

React Cache Usage
import { cache } from 'react';

// Automatic caching for server components
const fetchAsset = cache(async (nodeId: string) => {
  // Fetch implementation with automatic deduplication
});

Cache Strategies:

  • React Server Component cache deduplication
  • Drupal cache tag invalidation
  • Edge config cache synchronization
  • Feature flag cache warming

2. Error Handling and Resilience​

Robust Error Handling
import { responseCreate } from '@schwab/utilities/functions/responseCreate';
import { runtimeError } from '@schwab/utilities/functions/runtimeError';
import { SchLogger } from '@schwab/utilities/SchLogger';

// Standardized error response pattern
if (!fetchedResponse.isOk) {
  SchLogger.error('API fetch failed', { 
    service: 'drupal',
    endpoint: url,
    error: fetchedResponse.error 
  });
  return responseCreate(false, null, fetchedResponse.error);
}

3. Type Safety with Zod Validation​

Runtime Type Validation
import { z } from 'zod';
import { FetchedDrupalAssetSchema } from '@schwab/schema/fetched/cmap-api/FetchedDrupalAssetSchema';

const schema = z.array(FetchedDrupalAssetSchema).nonempty();
const zodResponse = schema.safeParse(fetchedResponse.data);

if (!zodResponse.success) {
  return responseCreate(false, null, zodResponse.error);
}

Service-Specific Implementations​

Salesforce Marketing Cloud (SFMC)​

Location: src/sfmc/

SFMC Integration
export { postLeadData } from './src/sfmc/postLeadData.ts';

Features:

  • Lead data synchronization
  • Marketing campaign integration
  • Customer journey tracking

Google Services Integration​

Location: src/google/

  • Analytics data retrieval
  • Tag Manager integration
  • Custom dimension tracking

RRBUS Content Platform​

Location: src/rrbus/

  • Content delivery network integration
  • Asset optimization
  • Performance monitoring

Utility Functions​

1. Cache Management Utilities​

Location: src/utilities/

Cache Utilities
// Cache revalidation and tag management
export { revalidateByTag } from './src/utilities/revalidateByTag.ts';
export { revalidateByPath } from './src/utilities/revalidateByPath.ts';
export { getCacheKey } from './src/utilities/getCacheKey.ts';

2. Response Standardization​

Standardized Response Pattern
export type TResponse<T> = {
  isOk: boolean;
  data: T;
  error?: Error | string;
  cacheTags?: string[];
  revalidate?: number;
};

3. Data Source Management​

Dynamic Data Source Configuration
import { getDataSourceUrl } from '@schwab/utilities/functions/getDataSourceUrl';

// Environment-aware API endpoint resolution
const dataSourceUrl = getDataSourceUrl(EDrupalSource.Education);

Development Patterns​

1. Service Factory Pattern​

Service Factory Implementation
// Base fetch function for service-specific implementations
function createServiceFetcher<T>(
  baseUrl: string,
  defaultOptions: RequestInit = {}
) {
  return async function<R>(
    endpoint: string, 
    options?: RequestInit
  ): Promise<TResponse<R>> {
    // Implementation with error handling, caching, validation
  };
}

export const drupalFetch = createServiceFetcher(process.env.DRUPAL_BASE_URL);
export const bynderFetch = createServiceFetcher(process.env.BYNDER_BASE_URL);

2. Transformer Integration​

Data Transformation Pipeline
import { transformDrupalAsset } from '@schwab/transformer/transformDrupalAsset';

// Fetch → Validate → Transform → Return
const rawData = await drupalFetch(endpoint);
const validatedData = schema.parse(rawData);
const transformedData = transformDrupalAsset(validatedData, role);

return responseCreate(true, transformedData);

Performance Optimization​

1. Request Deduplication​

  • React cache integration for automatic deduplication
  • Request coalescing for identical API calls
  • Smart cache invalidation strategies

2. Edge Computing Integration​

  • Vercel Edge Config for ultra-low latency
  • Geographic distribution of cached data
  • Smart cache warming algorithms

3. Parallel Processing​

Parallel Request Processing
// Concurrent API calls with Promise.all
const [asset, relatedContent, taxonomy] = await Promise.all([
  getDrupalAsset(nodeId),
  getRelatedContent(nodeId),
  getTaxonomyList(categoryId)
]);

Testing and Mocking​

Mock Data Integration​

Testing Support
import { createMockResponse } from '@schwab/mock-data';

// Development and testing mock responses
export const getMockDrupalAsset = () => 
  createMockResponse(mockDrupalAssetData);

Security and Compliance​

1. Authentication Management​

  • API key rotation and management
  • OAuth token handling
  • Service-to-service authentication

2. Data Privacy​

  • PII handling compliance
  • GDPR data processing
  • Audit logging for sensitive operations

3. Rate Limiting​

Rate Limiting Implementation
// Built-in rate limiting and retry logic
const rateLimitedFetch = withRateLimit(baseFetch, {
  maxRequests: 100,
  windowMs: 60000,
  retryAfter: 1000
});

Monitoring and Observability​

1. Logging Integration​

Structured Logging
import { SchLogger } from '@schwab/utilities/SchLogger';

SchLogger.info('API request initiated', {
  service: 'drupal',
  endpoint: '/api/cmap/asset',
  nodeId,
  duration: requestTime
});

2. Performance Metrics​

  • API response time tracking
  • Cache hit/miss ratios
  • Error rate monitoring
  • Data freshness metrics

Future Enhancements​

Planned Improvements​

  1. GraphQL Integration: Unified GraphQL layer over REST APIs
  2. Real-time Updates: WebSocket integration for live data
  3. Advanced Caching: Intelligent cache prefetching and warming
  4. Multi-region Support: Geographic data distribution
  5. Enhanced Security: Advanced authentication and authorization

Integration Roadmap​

  • Microservices: Service mesh integration
  • Event Streaming: Kafka/Pub-Sub integration
  • AI/ML: Intelligent content recommendations
  • Analytics: Enhanced data analytics and reporting

The Fetch package provides a robust, type-safe, and performant foundation for all external API integrations across the Charles Schwab monorepo. Its standardized approach ensures consistency, reliability, and maintainability across all consuming applications.