C#•4y ago

Can't get items to be added to update

I can't seem to get the "update" List filled..
I have this Json data https://hastebin.com/bazadowese.json
What I am trying to do, is everything that is not in the targetJson should be added to the "inject" list, and the items that are in the targetJson should be checked against the items in the sourceJson, if they are not in there, done. If they are check if they have differences, if they have differences add that item to the "update" list.

Currently it just adds everything from the sourceJson into the "inject" list and doesn't do squat for the update list.

Here's the function:

string requestBody = await new StreamReader(req.Body).ReadToEndAsync();
LookupInputs data = JsonConvert.DeserializeObject<LookupInputs>(requestBody);
if (data == null)
    return new BadRequestObjectResult("There was no data provided, or the data was not formatted properly");

Dictionary<string, List<Dictionary<string, JToken>>> result = new()
{
    { "inject", new() },
    { "update", new() }
};

if (data.TargetJson.Count == 0)
{
    result["inject"] = (from item in data.SourceJson select item).ToList();
}
else
{
    result["inject"] = (from source in data.SourceJson
        from target in data.TargetJson
        where source[data.CommonIdentifier] != target[data.CommonIdentifier] && !source.DictEquals(target)
        select source).ToList();

    result["update"] = (from source in data.SourceJson
        from target in data.TargetJson
        where source[data.CommonIdentifier] == target[data.CommonIdentifier] // && !source.DictEquals(target)
        select target).ToList();
}

return new OkObjectResult(results);

string requestBody = await new StreamReader(req.Body).ReadToEndAsync();
LookupInputs data = JsonConvert.DeserializeObject<LookupInputs>(requestBody);
if (data == null)
    return new BadRequestObjectResult("There was no data provided, or the data was not formatted properly");

Dictionary<string, List<Dictionary<string, JToken>>> result = new()
{
    { "inject", new() },
    { "update", new() }
};

if (data.TargetJson.Count == 0)
{
    result["inject"] = (from item in data.SourceJson select item).ToList();
}
else
{
    result["inject"] = (from source in data.SourceJson
        from target in data.TargetJson
        where source[data.CommonIdentifier] != target[data.CommonIdentifier] && !source.DictEquals(target)
        select source).ToList();

    result["update"] = (from source in data.SourceJson
        from target in data.TargetJson
        where source[data.CommonIdentifier] == target[data.CommonIdentifier] // && !source.DictEquals(target)
        select target).ToList();
}

return new OkObjectResult(results);

string requestBody = await new StreamReader(req.Body).ReadToEndAsync();
LookupInputs data = JsonConvert.DeserializeObject<LookupInputs>(requestBody);
if (data == null)
    return new BadRequestObjectResult("There was no data provided, or the data was not formatted properly");

Dictionary<string, List<Dictionary<string, JToken>>> result = new()
{
    { "inject", new() },
    { "update", new() }
};

if (data.TargetJson.Count == 0)
{
    result["inject"] = (from item in data.SourceJson select item).ToList();
}
else
{
    result["inject"] = (from source in data.SourceJson
        from target in data.TargetJson
        where source[data.CommonIdentifier] != target[data.CommonIdentifier] && !source.DictEquals(target)
        select source).ToList();

    result["update"] = (from source in data.SourceJson
        from target in data.TargetJson
        where source[data.CommonIdentifier] == target[data.CommonIdentifier] // && !source.DictEquals(target)
        select target).ToList();
}

return new OkObjectResult(results);

string requestBody = await new StreamReader(req.Body).ReadToEndAsync();
LookupInputs data = JsonConvert.DeserializeObject<LookupInputs>(requestBody);
if (data == null)
    return new BadRequestObjectResult("There was no data provided, or the data was not formatted properly");

Dictionary<string, List<Dictionary<string, JToken>>> result = new()
{
    { "inject", new() },
    { "update", new() }
};

if (data.TargetJson.Count == 0)
{
    result["inject"] = (from item in data.SourceJson select item).ToList();
}
else
{
    result["inject"] = (from source in data.SourceJson
        from target in data.TargetJson
        where source[data.CommonIdentifier] != target[data.CommonIdentifier] && !source.DictEquals(target)
        select source).ToList();

    result["update"] = (from source in data.SourceJson
        from target in data.TargetJson
        where source[data.CommonIdentifier] == target[data.CommonIdentifier] // && !source.DictEquals(target)
        select target).ToList();
}

return new OkObjectResult(results);

this is the current result: https://hastebin.com/buxaroneda.json
the result I'm expecting is that the item "BookID": "6" is left out of the "inject" list, and added to the "update" list

becquerel•9/30/22, 10:57 AM

Have you confirmed that

data.TargetJson.Count

data.TargetJson.Count

is greater than 0?

malkavOP•9/30/22, 10:57 AM

yes, I've been logging all over the place

the

data.TargetJson.Count

data.TargetJson.Count

== 1

becquerel•9/30/22, 10:58 AM

ok - didn't think it would be that simple

might take me a min to parse your logic...

malkavOP•9/30/22, 10:58 AM

sorry if it's a mess to sort through

becquerel•9/30/22, 10:59 AM

something should be in update if:

it is in targetJson
is also in sourceJson
has differences to the corresponding item in sourceJson

becquerel•9/30/22, 10:59 AM

is that correct?

becquerel•9/30/22, 11:00 AM

if so, don't you want that !source.DictEquals comment added back in?

becquerel•9/30/22, 11:00 AM

just as a tip - there is an explicit

join

join

syntax in LINQ you can use to make it a bit cleaner

malkavOP•9/30/22, 11:01 AM

Let me "Try" and help

if (data.TargetJson.Count == 0)

if (data.TargetJson.Count == 0)

this part just adds all the items from

data.SourceJson

data.SourceJson

into the "inject" list of the results

now comes the fun part lol

result["inject"] = ...

result["inject"] = ...

is supposed to iterate through both

data.TargetJson

data.TargetJson

and

data.SourceJson

data.SourceJson

and return all the items from

data.SourceJson

data.SourceJson

that are not in

data.TargetJson

data.TargetJson

(though I see that it is returning the item if only one property of the item in the Dictionary is different because they won't equal, but whatever)

result["update'] = ...

result["update'] = ...

is supposed to return the items from

data.TargetJson

data.TargetJson

that have the same

CommonIdentifier

CommonIdentifier

as in

data.SourceJson

data.SourceJson

but have the rest of the properties different (so it needs updating)

becquerel•9/30/22, 11:01 AM

var x = from source in data.SourceJson
        join target in data.TargetJson on // blah
        select ...

var x = from source in data.SourceJson
        join target in data.TargetJson on // blah
        select ...

Bbecquerel if so, don't you want that !source.DictEquals comment added back in?

malkavOP•9/30/22, 11:01 AM

yea I was testing if that resulted in something different

Bbecquerel ```cs var x = from source in data.SourceJson join target in data.TargetJ...

malkavOP•9/30/22, 11:02 AM

not sure if I want to join, since I want the item to be pushed to a new List in the

results

results

variable? but I'm not sure what join would do in this line of code

becquerel•9/30/22, 11:02 AM

return all the items from data.SourceJson that are not in data.TargetJson

would

data.SourceJson.Except(data.TargetJson)

data.SourceJson.Except(data.TargetJson)

work here?

malkavOP•9/30/22, 11:03 AM

I have that in my

DictEquals

DictEquals

method
Wait, here's that method:

public static bool DictEquals<TKey, TValue>(this Dictionary<TKey, TValue> source, Dictionary<TKey, TValue> target) 
        => source.Count == target.Count && !source.Except(target).Any();

public static bool DictEquals<TKey, TValue>(this Dictionary<TKey, TValue> source, Dictionary<TKey, TValue> target) 
        => source.Count == target.Count && !source.Except(target).Any();

malkavOP•9/30/22, 11:03 AM

I tested, and that acutally does the Equals for me quite well so far

Mmalkav not sure if I want to join, since I want the item to be pushed to a new List in ...

becquerel•9/30/22, 11:03 AM

the join is equivalent to your thing of using two

from

from

s with a

where

where

on the commonidentifier

Bbecquerel the join is equivalent to your thing of using two `from`s with a `where` on the ...

malkavOP•9/30/22, 11:04 AM

Ah fair enough, so it pretty much would do the same but cleaner?

TeBeCo•9/30/22, 11:04 AM

i still don't get why the original code invole JToken and Dictionary

TeBeCo•9/30/22, 11:04 AM

¯\_(ツ)_/¯

TeBeCo•9/30/22, 11:04 AM

deserialize to C# POCO ... map / add to REAL list => reserialize

TTeBeCo **__i still don't get why the original code invole JToken and Dictionary__**

malkavOP•9/30/22, 11:04 AM

Because the JToken represents whatever data in the JSON object would be the value of the key.
The string is the key

TeBeCo•9/30/22, 11:04 AM

no JSON magic nowhere

TeBeCo•9/30/22, 11:04 AM

yeah, don't

TeBeCo•9/30/22, 11:04 AM

use POCO

TeBeCo•9/30/22, 11:04 AM

work on that

TeBeCo•9/30/22, 11:04 AM

simple C#

TeBeCo•9/30/22, 11:05 AM

your data is uniform

malkavOP•9/30/22, 11:05 AM

wait, lemme explain, and you can correct me on that

malkavOP•9/30/22, 11:06 AM

the example JSON I sent is not the data I will always get...
What I will receive is a JSON array of objects. These objects have keys (string) and values (unknown)
so it was either dynamic, JValue, JToken, or whichever other type of dynamic data because the values can be int, string, objects, or whatever data type

TeBeCo•9/30/22, 11:06 AM

public record Book(string BookID, string BookTitle, sintrg BookPrice);
public record WeirdJson(Book[] sourceJson, Book[] targetJson, string commonIdentifier);

var weirdStuff = STJ.Deserialize<WeirdJson>(inputJson);

// work with C# here

public record Book(string BookID, string BookTitle, sintrg BookPrice);
public record WeirdJson(Book[] sourceJson, Book[] targetJson, string commonIdentifier);

var weirdStuff = STJ.Deserialize<WeirdJson>(inputJson);

// work with C# here

malkavOP•9/30/22, 11:07 AM

No, no, the example is uniform but the data I will receive is not

TeBeCo•9/30/22, 11:07 AM

oO so you mean the code and data you sent is not like the actual one ?

malkavOP•9/30/22, 11:08 AM

the code is the actual code I intend to use, but the data I send is not the actual one.
the data I will receive is based on whatever the user needs to do a

Lookup Transformation

Lookup Transformation

malkavOP•9/30/22, 11:08 AM

but in SSIS and not in Informatica

malkavOP•9/30/22, 11:11 AM

What I'm finding so far is that

source[data.CommonIdentifier] == target[data.CommonIdentifier]

source[data.CommonIdentifier] == target[data.CommonIdentifier]

isn't returning a value. While I'm sure that in the example data I sent both have a

BookID

BookID

"6"

"6"

, so I'm a little confused why it won't return a value, not even in this loop:

foreach (Dictionary<string, JToken> source in data.SourceJson)
{
    log.LogDebug("Iterating through both dictionaries to check if the target even gets through");
    foreach (Dictionary<string, JToken> target in data.TargetJson.Where(target =>
                 source[data.CommonIdentifier] == target[data.CommonIdentifier]))
    {
        log.LogDebug("Target item: {T}, {S}", target[data.CommonIdentifier], source[data.CommonIdentifier]);
    }
}

foreach (Dictionary<string, JToken> source in data.SourceJson)
{
    log.LogDebug("Iterating through both dictionaries to check if the target even gets through");
    foreach (Dictionary<string, JToken> target in data.TargetJson.Where(target =>
                 source[data.CommonIdentifier] == target[data.CommonIdentifier]))
    {
        log.LogDebug("Target item: {T}, {S}", target[data.CommonIdentifier], source[data.CommonIdentifier]);
    }
}

TeBeCo•9/30/22, 11:12 AM

the code is the actual code I intend to use, but the data I send is not the actual one.

I mean if you send data that is not the why you'll use, the code is irrelevant

TeBeCo•9/30/22, 11:12 AM

proof being my suggestion to remove 99% of your code since it would have been useless

malkavOP•9/30/22, 11:12 AM

because I had to have some test data don't I

TeBeCo•9/30/22, 11:12 AM

¯\_(ツ)_/¯

TeBeCo•9/30/22, 11:13 AM

it's probably impossible to ask "how to parse data" is no one know the data format

TeBeCo•9/30/22, 11:13 AM

(or have the wrong format)

malkavOP•9/30/22, 11:14 AM

how would you go about this then?
I get three values from the user

source

source

target

target

, and

commonIdentifier

commonIdentifier

source and target are both an Array of objects
and commonIdentifier is the key to which the value should match

TeBeCo•9/30/22, 11:14 AM

if the data is inconsistent in your JSON then either ... it's trully random and yes you use JObject/JToken (if you use Newtonsoft)

TeBeCo•9/30/22, 11:14 AM

or it's not really random but more like inheritance

malkavOP•9/30/22, 11:14 AM

I use Newtonsoft

TeBeCo•9/30/22, 11:14 AM

and then you use json differenciator

TeBeCo•9/30/22, 11:14 AM

and you could then still use C#

Mmalkav I use Newtonsoft

TeBeCo•9/30/22, 11:15 AM

STJ also have support for JSON DOM API

TeBeCo•9/30/22, 11:15 AM

that;s all I mean

TeBeCo•9/30/22, 11:15 AM

Utf8JsonReader

Utf8JsonReader

Can't get items to be added to update

Similar Threads

Similar Threads

Similar Threads