i would suggest to use htmlagilitypack. because with that you can differentiate all very easily with
i would suggest to use htmlagilitypack. because with that you can differentiate all very easily with xpath as i said. //a and check href attribute
Fif url doesnt have http then it is relative
KWhen crawl toros univarsty website email+phone number come along they have the <a href""
I guess I should add some parameters to prevent it
I guess I should add some parameters to prevent it
Fyou can easily use URL class of c#
Fand check if valid url
Fhere a method written by chatgpt. i havent tested you can test :d
Fpublic static bool IsValidUrl(string rootUrl, string relativeUrl)
{
Uri rootUri;
if (!Uri.TryCreate(rootUrl, UriKind.Absolute, out rootUri))
{
return false;
}
Uri combinedUri;
if (!Uri.TryCreate(rootUri, relativeUrl, out combinedUri))
{
return false;
}
return combinedUri.IsAbsoluteUri;
}
{
Uri rootUri;
if (!Uri.TryCreate(rootUrl, UriKind.Absolute, out rootUri))
{
return false;
}
Uri combinedUri;
if (!Uri.TryCreate(rootUri, relativeUrl, out combinedUri))
{
return false;
}
return combinedUri.IsAbsoluteUri;
}
Ktested it it still return true for "mailto:info@toros.edu.tr" which shouldn't. couldnt find any useful method on google as the relative or absolute question comes up instead each time I google it and the ai gives some regx that don't work
Fi am checking now
Fvar rootUrl = "https://toros.edu.tr/";
var srValue = "mailto:info@toros.edu.tr";
Uri absolute = new Uri(rootUrl);
Uri result = new Uri(absolute, srValue);
srValue = result.AbsoluteUri;
Uri uriResult;
var resultgg = Uri.TryCreate(absolute, srValue, out uriResult)
&& (uriResult.Scheme == Uri.UriSchemeHttp || uriResult.Scheme == Uri.UriSchemeHttps);
var srValue = "mailto:info@toros.edu.tr";
Uri absolute = new Uri(rootUrl);
Uri result = new Uri(absolute, srValue);
srValue = result.AbsoluteUri;
Uri uriResult;
var resultgg = Uri.TryCreate(absolute, srValue, out uriResult)
&& (uriResult.Scheme == Uri.UriSchemeHttp || uriResult.Scheme == Uri.UriSchemeHttps);
Fthis one works i tested
M
Fhello welcome. checking now
Fok it says that you didnt install scheduler library
Fhave you executed these code blocks?
F
F
Mhello thank you very much
Mdo i install them by running the install requirements ? i did this
Fyes but you need to do it after each time your session ends you start again
Fthey are not permanently staying
Myes i understand
Mwhen running i see this
M
FI can connect and check your computer if you wish. google remote any desk etc if you have
Fcan you execute this one
F
Fwhat does it show after you get green check mark
M
Fthat is good
Fnow execute this and show me results please
F
Fyou need to get green check mark in each cell
Mhere you go
Fis full message like this
Mno not at all
Monly what i showed you
Fok that is the problem
Fcan you shut down your instance
Fand start again?
Mokay i see
Mhow do i shut down the instance?
F
Fhere
Mthanks
Mdone
Mthen i reconnect?
Fyep
Myess it worked
Fhave you done training part?
Myes training has been done before