CA
Crawlee & Apify•12mo ago
like-gold

Output schema

Hi, I'm having 2 issues with output schema definition: - I can't make transformations.unwind work, the table only shows undefined values -> Is there a working example I can check somewhere? - When instead using transformations.flatten, it seems the table can't show fields with array or object format, it shows undefined instead. Am I missing something here? For reference, my result is formatted like this, and i want to unwind/flatten the data part: 
[
  {
    "metadata": {
      "timestamp": "2024-07-08T09:51:31.942Z",
      "run_id": "lWmckTaBAlbeKTM33"
    },
    "data": {
      "url": "some_url.com",
      "title": "some title",
      "attributes": ["a", "b"]
    }
  },
  {...}
]
[
  {
    "metadata": {
      "timestamp": "2024-07-08T09:51:31.942Z",
      "run_id": "lWmckTaBAlbeKTM33"
    },
    "data": {
      "url": "some_url.com",
      "title": "some title",
      "attributes": ["a", "b"]
    }
  },
  {...}
]
And I'd like the table to show the columns: - "metadata" as object (this works) - "url" as link (works only when using flatten, not unwind) - "title" as text (same) - "attributes" as array (shows undefined) Here is the docs I've been using https://docs.apify.com/platform/actors/development/actor-definition/output-schema
7 Replies
optimistic-gold
optimistic-gold•12mo ago
Please provide your exact reproduction (JSON with output_schema). It should be straightforward and shouldn't cause any issues. The documentation is clear and simple, so there might be a mistake in your JSON structure.
like-gold
like-goldOP•12mo ago
Hi @Oleg V. , thanks for your answer. Here is my output definition: 
{
    "actorSpecification": 1,
    "fields": {},
    "views": {
        "price_monitor": {
            "title": "Price Monitor",
            "description": "`PriceMonitor` output schema.",
            "transformation": {
                "fields": [
                    "metadata",
                    "data.url",
                    "data.id",
                    "data.title",
                    "data.brand",
                    "data.price"
                ],
                "flatten": [
                    "data"
                ]
            },
            "display": {
                "component": "table",
                "properties": {
                    "metadata": {
                        "format": "object"
                    },
                    "data.url": {
                        "label": "URL",
                        "format": "link"
                    },
                    "data.id": {
                        "label": "ID",
                        "format": "text"
                    },
                    "data.title": {
                        "label": "Title",
                        "format": "text"
                    },
                    "data.brand": {
                        "label": "Brand",
                        "format": "text"
                    },
                    "data.price": {
                        "label": "Price",
                        "format": "object"
                    }
                }
            }
        }
    }
}
{
    "actorSpecification": 1,
    "fields": {},
    "views": {
        "price_monitor": {
            "title": "Price Monitor",
            "description": "`PriceMonitor` output schema.",
            "transformation": {
                "fields": [
                    "metadata",
                    "data.url",
                    "data.id",
                    "data.title",
                    "data.brand",
                    "data.price"
                ],
                "flatten": [
                    "data"
                ]
            },
            "display": {
                "component": "table",
                "properties": {
                    "metadata": {
                        "format": "object"
                    },
                    "data.url": {
                        "label": "URL",
                        "format": "link"
                    },
                    "data.id": {
                        "label": "ID",
                        "format": "text"
                    },
                    "data.title": {
                        "label": "Title",
                        "format": "text"
                    },
                    "data.brand": {
                        "label": "Brand",
                        "format": "text"
                    },
                    "data.price": {
                        "label": "Price",
                        "format": "object"
                    }
                }
            }
        }
    }
}
As you see (first image) the price should be an object but is instead undefined, even though the price json is present in the full data (second image).
No description
No description
like-gold
like-goldOP•12mo ago
If i try using unwind with this config: 
{
    "actorSpecification": 1,
    "fields": {},
    "views": {
        "price_monitor": {
            "title": "Price Monitor",
            "description": "`PriceMonitor` output schema.",
            "transformation": {
                "fields": [
                    "metadata",
                    "url",
                    "id",
                    "title",
                    "brand",
                    "price"
                ],
                "flatten": [
                    "unwind"
                ]
            },
            "display": {
                "component": "table",
                "properties": {
                    "metadata": {
                        "format": "object"
                    },
                    "url": {
                        "label": "URL",
                        "format": "link"
                    },
                    "id": {
                        "label": "ID",
                        "format": "text"
                    },
                    "title": {
                        "label": "Title",
                        "format": "text"
                    },
                    "brand": {
                        "label": "Brand",
                        "format": "text"
                    },
                    "price": {
                        "label": "Price",
                        "format": "object"
                    }
                }
            }
        }
    }
}
{
    "actorSpecification": 1,
    "fields": {},
    "views": {
        "price_monitor": {
            "title": "Price Monitor",
            "description": "`PriceMonitor` output schema.",
            "transformation": {
                "fields": [
                    "metadata",
                    "url",
                    "id",
                    "title",
                    "brand",
                    "price"
                ],
                "flatten": [
                    "unwind"
                ]
            },
            "display": {
                "component": "table",
                "properties": {
                    "metadata": {
                        "format": "object"
                    },
                    "url": {
                        "label": "URL",
                        "format": "link"
                    },
                    "id": {
                        "label": "ID",
                        "format": "text"
                    },
                    "title": {
                        "label": "Title",
                        "format": "text"
                    },
                    "brand": {
                        "label": "Brand",
                        "format": "text"
                    },
                    "price": {
                        "label": "Price",
                        "format": "object"
                    }
                }
            }
        }
    }
}
then it's all undefined
No description
optimistic-gold
optimistic-gold•12mo ago
unwind usage should look like this: 
 
                 "transformation": {
                      "fields": [
                          "searchQuery",
                          "resultsTotal",
                          "organicResults"
                      ],
                      "unwind": [
                          "organicResults"
                      ]
                  },
 
                 "transformation": {
                      "fields": [
                          "searchQuery",
                          "resultsTotal",
                          "organicResults"
                      ],
                      "unwind": [
                          "organicResults"
                      ]
                  },
And not 
                "flatten": [
                    "unwind"
                ]
                "flatten": [
                    "unwind"
                ]
try to debug it with smaller / simpler output object. and then increase complexity step by step (I mean add nested obj, and so on...). check this working example for inspiration: 
        "views": {
          "overview": {
            "title": "Overview",
            "description": "",
            "transformation": {
              "fields": [
                "thumbnailImage",
                "asin",
                "title",
                "description",
                "brand",
                "stars",
                "reviewsCount",
                "price.value",
                "breadCrumbs",
                "url"
              ],
              "flatten": [
                "price"
              ]
            },
            "display": {
              "component": "table",
              "properties": {
                "thumbnailImage": {
                  "label": "Image",
                  "format": "image"
                },
                "asin": {
                  "label": "ASIN"
                },
                "description": {
                  "format": "text"
                },
                "stars": {
                  "format": "number"
                },
                "reviewsCount": {
                  "format": "number"
                },
                "price.value": {
                  "label": "Price",
                  "format": "number"
                },
                "breadCrumbs": {
                  "label": "Categories"
                },
                "url": {
                  "label": "URL",
                  "format": "link"
                }
              }
            }
          }
        }
        "views": {
          "overview": {
            "title": "Overview",
            "description": "",
            "transformation": {
              "fields": [
                "thumbnailImage",
                "asin",
                "title",
                "description",
                "brand",
                "stars",
                "reviewsCount",
                "price.value",
                "breadCrumbs",
                "url"
              ],
              "flatten": [
                "price"
              ]
            },
            "display": {
              "component": "table",
              "properties": {
                "thumbnailImage": {
                  "label": "Image",
                  "format": "image"
                },
                "asin": {
                  "label": "ASIN"
                },
                "description": {
                  "format": "text"
                },
                "stars": {
                  "format": "number"
                },
                "reviewsCount": {
                  "format": "number"
                },
                "price.value": {
                  "label": "Price",
                  "format": "number"
                },
                "breadCrumbs": {
                  "label": "Categories"
                },
                "url": {
                  "label": "URL",
                  "format": "link"
                }
              }
            }
          }
        }
like-gold
like-goldOP•12mo ago
Ofc my bad for the typo, I did use "unwind" as a key, but still could not make it work. If i do "unwing": ["data"], and data contains a url field, what should i list in the fields? I've tried url and data.url adn both end up undefined. In your example with flatten none of the fields are of type array or object (they are image, text or number only), which are the types I'm having trouble with...
like-gold
like-goldOP•12mo ago
So with unwind I'm able to move children into a parent object as expected, but in the table view it always appears as undefined. What should be in the field list when using unwind?
No description
No description
No description
like-gold
like-goldOP•11mo ago
Any insight on this @Oleg V.?

Did you find this page helpful?