Intrest in support for rpc types?

[vstreame]

I was wondering if typical is interest in adding a third top level type (in addition to struct and choice), rpc. That way one can model their APIs in typical along side their data models. For example:

struct User {
  id: String = 0
  username: String = 1
}

struct CreateUserInput {
  user: User = 0
}

choice CreateUserOutput {
  success = 0
  error: String = 1
}

rpc UserService {
  createUser(CreateUserInput): CreateUserOutput
}

I can imagine asymmetric also being useful for rpc as a way to deprecate API endpoints. Where the server must implement it, but the clients must handle the case where it is gone.

rpc UserService {
  # Deprecated: Use createUserV2 instead
  asymmetric createUser(CreateUserInput): CreateUserOutput
  createUserV2(CreateUserInputV2): CreateUserOutput
}

Or does RPC just open up a whole can of worms that typical doesn't want to deal with? My thinking is that Typical wouldn't handle the actual doing of the rpc, but the code generation would provide hooks for userland to fill in. That way the rpc could be through any user-defined protocol.

[stepchowfun]

Thank you @vstreame for opening this issue! One of the main use cases for Typical is to encode/decode RPC messages, so I think this request makes a lot of sense. However, I can also see this as being outside the scope of Typical, which is a general serialization/deserialization framework and not specific to RPCs.

The idea of using asymmetric to deprecate RPC endpoints is interesting!

For now, I'd like to leave this issue open for further discussion. I can't promise that this will get implemented, but I'd like to consider it at least. I think it would be helpful to see an example of what the generated code (an interface that would be implemented by the user?) would look like for both the client and the server.

[vstreame]

@stepchowfun Totally makes sense. This is what I could imagine the generated code would look like using a type of strategy pattern to inject the transportation protocol. I'm a bit better at Typescript than Rust so I hope you don't mind the code being in TS 😅 Also I haven't tested this code so I'm sure there are small mistakes here and there.

//  generated TS rpc type
export type Fetcher = (rpcName: string, inputData: ArrayBuffer) => Promise<ArrayBuffer>;

export class UserService {
  constructor(private fetcher: Fetcher) {
  }

  async createUser(input: CreateUserInput): Promise<CreateUserOutput> {
    // 1. Serialize input into raw bytes
    const inputData = CreateUserInput.serialize(input);
    // 2. Send raw bytes using provided strategy
    const payloadData = await this.fetcher('createUser', inputData);
    // 3. Deserialize raw response
    const message = CreateUserOutput.deserialize(payloadData);
    // 4. Error checking
    if (message instanceof Error) throw message;
    // 5. Return deserialized object
    return message;
  }
}

Once we have this pretty simple generated code developers just need to write the strategy that works best for their own systems. Here's some examples:

import { type Fetcher, UserService } from './generated/types';

// https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API
const fetchStrategy: Fetcher = async (rpcName, inputData) => {
  const baseURL = 'https://my.company.user.service';
  const response = await fetch(`${baseURL}/rpc/${rpcName}`, {
    method: 'POST',
    body: inputData,
    headers: {
      // Any authentication headers used by my company/org
    },
  });
  if (!response.ok) throw new Error('RPC request failed');
  return response.arrayBuffer();
};

// https://github.com/nats-io/nats.js
define const nc: NatsConnection;
const natsStrategy: Fetcher = async (rpcName, inputData) => {
  const message = await nc.request(`users.rpc.${rpcName}`, new Uint8Array(inputData), { timeout: 1000 });
  return message.data.buffer;
};

// A strategy used for testing where no external request is made
const testPayloads: Record<string, ArrayBuffer> = {
  createUser: CreateUserOutput.serialize({ $field: 'success' });
};
const testStrategy: Fetcher = (rpcName, _inputData /* ignored */) => {
  return Primise.resolve(testPayloads[rpcName]);
}

// Note: These all use classes & constructors, but you could also change the API to 
//            use closures instead if that would look nicer
const userServiceUsingFetch = new UserService(fetchStrategy);
const userServiceUsingNATS = new UserService(natsStrategy);
const userServiceForTests = new UserService(testStrategy);