arktype•2mo ago

Structure Node hinder DeepPartial

Hey, just stumbled across this amazing community and thought you might be able to help me. i was using a modified version of the deepPartial Function, that has been posted here https://discord.com/channels/957797212103016458/957804102685982740/1348929382261067777 I have stumbled into an issue with certain type definitions:

const loanGroupSchema = type({
      
      hasLoans:type
  .or(
    {
      loanType: type("string"),
      loanDetails: {
        bankName: type("string"),
        payoutDate: type("string"),
      },
      unionBranchMarker: "'fullInformationRequired'",
    },
    {
      loanType: type("string"),
      loanAmount: type("number"),
      unionBranchMarker: "'reducedInformation'",
    },
  ))}

existingLoans: type({ "[string]": loanGroupSchema }) <- this one is the problem

const loanGroupSchema = type({
      
      hasLoans:type
  .or(
    {
      loanType: type("string"),
      loanDetails: {
        bankName: type("string"),
        payoutDate: type("string"),
      },
      unionBranchMarker: "'fullInformationRequired'",
    },
    {
      loanType: type("string"),
      loanAmount: type("number"),
      unionBranchMarker: "'reducedInformation'",
    },
  ))}

existingLoans: type({ "[string]": loanGroupSchema }) <- this one is the problem

while traversing the existing Loans node i come across an intersection that has no props. Although the Index still lies ahead.

{
  "key": "existingLoans",
  "value": {
    "index": [
      {
        "signature": "string",
        "value": {
          "required": [
            {
              "key": "loan",
              "value": {
                "branches": [
                  {
                    "required": [
...

{
  "key": "existingLoans",
  "value": {
    "index": [
      {
        "signature": "string",
        "value": {
          "required": [
            {
              "key": "loan",
              "value": {
                "branches": [
                  {
                    "required": [
...

now i am wondering, if my approach is wrong. while looking through the repo, i saw, that a structure only has props, if it has children that are optional or required. Maybe that is the source.

13 Replies

Carl_c24OP•2mo ago

message.txt

ssalbdivad•2mo ago

@Carl_c24 Ahh okay yes index signatures are not properties and can't really be made optional in a pure sense. If the version you worked on handles your use case well that's great- it looks like you went pretty deep into the type system to get everything working 🤓 🎉

Carl_c24OP•2mo ago

Sadly it does not, as of now it just fails to go deeper. That is why i am looking for a different way to reach those nodes. Any ideas?

ssalbdivad•2mo ago

This is getting into pretty complex territory with manipulating the internal structure of the type system. I don't have time to write the whole thing for you, but I would check out this method which we use internally for recursive transformations like this: https://github.com/arktypeio/arktype/blob/2ebcfce89aa071f06322fd4249cb78cc1602a457/ark/schema/node.ts#L492 You can access it like myType.internal.transform. Then you would want to write a rule that recurses looking for structure nodes, which could convert all the required props on that node to optional props while also maintaining the existing index signatures/sequences.

Carl_c24OP•2mo ago

thanks a lot, i will try this approach and look how far i get. I am just a bit confused as to the form of the mapper i shall provide.

const transformed = o.internal.transform( (kind, inner, ctx) => {
     if ("required" in inner) {
       const requiredValue = inner.required;
       const { required, ...rest } = inner;
       const optional = requiredValue?.map((required) => {
        return {
          ...required.inner,
        };
       });
       return  {...rest, optional: optional};
     }
     return {...inner};
   }, {shouldTransform: node => node.includesTransform});

const transformed = o.internal.transform( (kind, inner, ctx) => {
     if ("required" in inner) {
       const requiredValue = inner.required;
       const { required, ...rest } = inner;
       const optional = requiredValue?.map((required) => {
        return {
          ...required.inner,
        };
       });
       return  {...rest, optional: optional};
     }
     return {...inner};
   }, {shouldTransform: node => node.includesTransform});

this is the solution i came up with so far, it seems to be working as intended, but runs into parserErrors on a union of two branches.

Uncaught (in promise) ParseError: An unordered union of a type including a morph and a type with overlapping input is indeterminate:
Left: { loanAmount?: (In: number % 1) => Out<unknown>, loanType?: "agreed_overdraft_loan" | "balloon_loan" | "car_loan" | "consumer_loan" | "credit_card" | "employer_loan" | "flex_loan" | "leasing" | "loan_on_demand" | "zero_percent_financing", unionBranchMarker?: "reducedInformation" }
Right: { loanDetails?: { bankName?: string >= 1, estimatedRemainingDebt?: (In: number % 1) => Out<unknown>, fromGermanBank?: (In: "no" | "yes") => Out<unknown>, loanAmount?: (In: number % 1) => Out<unknown>, loanDuration?: (In: /^(?:(?!^-0\.?0*$)(?:-?(?:(?:0|[1-9]\d*)(?:\.\d+)?)|\.\d+?))$/) => Out<number>, monthlyRate?: (In: number % 1) => Out<unknown>, payoutDate?: (In: /^\d{2}\/\d{4}$/) => To<unknown> }, loanType?: "agreed_overdraft_loan" | "balloon_loan" | "car_loan" | "consumer_loan" | "credit_card" | "employer_loan" | "flex_loan" | "leasing" | "loan_on_demand" | "zero_percent_financing", unionBranchMarker?: "fullInformationRequired" }

Uncaught (in promise) ParseError: An unordered union of a type including a morph and a type with overlapping input is indeterminate:
Left: { loanAmount?: (In: number % 1) => Out<unknown>, loanType?: "agreed_overdraft_loan" | "balloon_loan" | "car_loan" | "consumer_loan" | "credit_card" | "employer_loan" | "flex_loan" | "leasing" | "loan_on_demand" | "zero_percent_financing", unionBranchMarker?: "reducedInformation" }
Right: { loanDetails?: { bankName?: string >= 1, estimatedRemainingDebt?: (In: number % 1) => Out<unknown>, fromGermanBank?: (In: "no" | "yes") => Out<unknown>, loanAmount?: (In: number % 1) => Out<unknown>, loanDuration?: (In: /^(?:(?!^-0\.?0*$)(?:-?(?:(?:0|[1-9]\d*)(?:\.\d+)?)|\.\d+?))$/) => Out<number>, monthlyRate?: (In: number % 1) => Out<unknown>, payoutDate?: (In: /^\d{2}\/\d{4}$/) => To<unknown> }, loanType?: "agreed_overdraft_loan" | "balloon_loan" | "car_loan" | "consumer_loan" | "credit_card" | "employer_loan" | "flex_loan" | "leasing" | "loan_on_demand" | "zero_percent_financing", unionBranchMarker?: "fullInformationRequired" }

Thank you for pointing me in the direction of the internal.transform. That was quite the interesting deep dive, but sadly i am back to being clueless. could this be the same issue as here: https://discord.com/channels/957797212103016458/1410375271483179039/1410375271483179039 @ssalbdivad Can you help me?

linus.eing•2mo ago

Hey, I think I have the same problem, is there any progress with this? @Carl_c24 @ArkDavid

Carl_c24OP•2mo ago

@linus.eing i tried a bunch of different stuff, but sadly i haven't gotten it to work yet. I am a bit confused about the internal Resolution of the Union Nodes. No luck so far, was hoping for pointers from @ArkDavid

ssalbdivad•2mo ago

@Carl_c24 can you post a full repro with the transform you wrote, the types you're using it on and the expected/actual output? Could be a playground link but as long as there is no missing code I will take a look: https://arktype.io/playground

ArkType

ArkType Playground

TypeScript's 1:1 validator, optimized from editor to runtime

Carl_c24OP•2mo ago

Thanks a lot for revisiting the issue. i tried to write out all relevant bits in the playground, but it doesn't like me defining typescript types. i placed the same code in my specs and it runs, there, so i hope you don't have too many issues running it. The intention in this is, to allow a form to be submitted in an incomplete state, while still transforming the values and handeling all that good stuff. to determine, which fields have to remain required, we use discriminator hints, but i honestly would just hope for the optionalize to work. We can determine the specific behaviour based on the key of a prop later. Thanks again @ArkDavid

Carl_c24OP•2mo ago

message.txt

ssalbdivad•2mo ago

@Carlc24 What you wrote looks pretty close, here is the version I landed on: ```ts function applyDeepPartialSchemaRules<o extends type<object | null | undefined>>( o: o ): type<RecursivePartial<o["t"]> | null | undefined> function applyDeepPartialSchemaRules( o: type<Record<string, unknown> | unknown[]> ): unknown { const transformed = o.internal.transform( (kind, inner, ctx) => { if (kind == "structure" && "required" in inner) { const optional: Prop.Schema[] = inner.optional ? [...inner.optional] : [] if (inner.required) { optional.push(...inner.required.map( => _.inner)) } return { ...inner, optional, required: [] } } return inner }, { shouldTransform: (node, ctx) => node.includesTransform } ) return transformed } ``` The error you get currently is because of this (see the note under unions): https://arktype.io/docs/expressions#union This wasn't occuring when all the properties were required because we could reliably tell them apart. You could either add a required descriminator back in or use something undeclaredKeys to specify that extra keys should be rejected which would also distinguish your branches and allow this expression: https://arktype.io/docs/objects#properties-undeclared

ArkType

ArkType Docs

TypeScript's 1:1 validator, optimized from editor to runtime

ArkType

ArkType Docs

TypeScript's 1:1 validator, optimized from editor to runtime

Carl_c24OP•2mo ago

You are an absolute wizard. Thank you so much. i managed to make it work like this:

export function applyDeepPartialSchemaRules<o extends type<object | null | undefined>>(
  o: o,
): type<RecursivePartial<o["t"]> | null | undefined>;
export function applyDeepPartialSchemaRules(o: type<Record<string, unknown> | unknown[]>): unknown {
  const transformed = o.internal.transform(
    (kind, inner) => {
        if (kind == "intersection" && "structure" in inner) {
          const discriminatorHint = inner.meta?.discriminatorHint;
          console.log("discriminatorHint", discriminatorHint)
            const optional: Prop.Schema[] =
                inner.structure?.optional ? [...inner.structure.optional] : []
            const required: Prop.Schema[] =
                inner.structure?.required ? [...inner.structure.required.filter(_ => _.key === discriminatorHint)] : []
            if (inner.structure?.required) {
                optional.push(...inner.structure.required
                    .filter(_ => _.key !== discriminatorHint)
                    .map(_ => _.inner))
            }
            return { ...inner, structure: { ...inner.structure?.inner, optional, required } }
        }
        return inner
    },
    { shouldTransform: (node, ctx) => node.includesTransform }
)
return transformed

export function applyDeepPartialSchemaRules<o extends type<object | null | undefined>>(
  o: o,
): type<RecursivePartial<o["t"]> | null | undefined>;
export function applyDeepPartialSchemaRules(o: type<Record<string, unknown> | unknown[]>): unknown {
  const transformed = o.internal.transform(
    (kind, inner) => {
        if (kind == "intersection" && "structure" in inner) {
          const discriminatorHint = inner.meta?.discriminatorHint;
          console.log("discriminatorHint", discriminatorHint)
            const optional: Prop.Schema[] =
                inner.structure?.optional ? [...inner.structure.optional] : []
            const required: Prop.Schema[] =
                inner.structure?.required ? [...inner.structure.required.filter(_ => _.key === discriminatorHint)] : []
            if (inner.structure?.required) {
                optional.push(...inner.structure.required
                    .filter(_ => _.key !== discriminatorHint)
                    .map(_ => _.inner))
            }
            return { ...inner, structure: { ...inner.structure?.inner, optional, required } }
        }
        return inner
    },
    { shouldTransform: (node, ctx) => node.includesTransform }
)
return transformed

i am using discriminator hints. I don't know why, but the metainformation is not provided to the structure node, so i have to stop a level above that and take it from the parent of the structure node. But alas, it works and it works great. Thanks a lot @ssalbdivad

ssalbdivad•2mo ago

Awesome, glad I could help!

Gaming

Programming

Structure Node hinder DeepPartial

Did you find this page helpful?