[URL] Found a URL with missing part Solved

Question

Hello,  I am looking to find a URL among a number of possibilities, knowing that only one is correct.  Therefore, I first need a way to create a list of URLs. Then I need a way to test them to see which one is correct.  For example: www.site.fr/rubrique/xyz.htm where x, y, and z are three letters that can be identical.  Could someone please help me?

lami20j · Accepted Answer

Re,  Thank you very much, it works but it may take some time  Yes, I know, you have almost 2 million URLs.  You can split the file for example. It's possible directly in the script.  So creating a file for each category will simplify things a bit.  It's up to you. If you need anything, let me know and I will make the necessary changes, but not tonight.  lami20j

Ssylvainsab · Answer

Good evening.  I think you could use regex with preg_match (check).  See you later -- Sylvain

Powo2 · Answer

What do you mean? I'm more than a beginner lol, I was rather hoping that there were already existing software. I've never programmed and I don't even know how to do it.  Is there not a software capable of creating a logical list and another capable of reading a list of URL addresses and determining which ones return data?

Ssylvainsab · Answer

But can you concretely say what you would like?  "Create a logical list:" Make a list of the URLs that may exist? For example, "www.commentcamarche.net" would be recognized as existing, but not blabla.boot?  "determine which ones return data" ??? Right now it’s very vague, I don’t understand at all. What do you mean?  See you later -- Sylvain

Powo2 · Answer

For example, I have an address:  www.adresse.fr/xyz.htm  I would like to find the x, y, and z that form an existing address (knowing that x, y, and z are 3 letters). So I would like the program to tell me:  www.adresse.fr/abc.htm does not exist www.adresse.fr/abd.htm does not exist www.adresse.fr/abe.htm does not exist www.adresse.fr/abf.htm exists  But I can't type the addresses one by one; there are too many...

Ssylvainsab · Answer

Ah ! Alright!  You want to check if the URL, or rather the corresponding file, exists. (and only check if it exists if it has 3 letters) Is that correct?  And you want to do this for which addresses (those from a particular domain?), and for how many? -- Sylvain

Powo2 · Answer

Here it is basically...  Actually, I have the beginning of my address but I'm missing 3 letters... and there are many combinations of 3 letters possible, that's why I would like a program that makes a list (I think the number of possibilities is on the order of 26^3 = 12167).  So I would like to know which one of these 12167 addresses is the right one...

Ssylvainsab · Answer

If I understood correctly, you have this address: http://www.site.com/abcdxxx.html  and you know all the letters except xxx. That's right?  For that, you'd need to make a regex in PHP (I can handle that, it's super easy :D), but then you'd need to list the possible URLs, and I'm not quite sure how to do that... Maybe with an array.  Finally, I have a few ideas in mind.  Give me the URL you know, and I'll make the regex that will check it. Then I'll try different scripts and see if any of them work.  See you later -- Sylvain

Powo2 · Answer

The address I know is:  http://feuks.free.fr/colles/colles_physio/P106xxx.rar  And I don't know the 3 x  But the problem is that I would like a program that can work multiple times ^^ not just on this address, for example I would also like to find the 3 x of this address:  http://feuks.free.fr/colles/colles_physio/P107xxx.rar

Ssylvainsab · Answer

Ah.  But how many different files would you like it to work for (without the xxx)? -- Sylvain

Powo2 · Answer

About a hundred ^^ if it were just two I wouldn’t have had a problem... but now...  That's why I wanted a program because then I could have managed on my own...

Ssylvainsab · Answer

And are the files all in the folder http://feuks.free.fr/colles/colles_physio/?  Do they have the same names with a number increased by 1 each time? Like: http://feuks.free.fr/colles/colles_physio/P106xxx.rar http://feuks.free.fr/colles/colles_physio/P107xxx.rar http://feuks.free.fr/colles/colles_physio/P108xxx.rar  etc...  Finally, give as many details as possible, everything you know. -- Sylvain

Powo2 · Answer

It goes up to P113 and then it's different address beginnings each time.  http://feuks.free.fr/colles/colles_anat/anat1xxx.rar http://feuks.free.fr/colles/colles_bio/b110xxx.rar http://feuks.free.fr/colles/colles_bioch/B104xxx.rar http://feuks.free.fr/colles/colles_biophys/b106xxx.rar  It's quite random. I have to admit these are paid training MCQs (in medicine)... and I don't have enough money to buy them all... after paying for a few, I noticed a redundancy... so I suppose these are addresses of this type with random letters.

Ssylvainsab · Answer

I'm starting to write a script for you, but it's not at all right (I'm searching for files with regex when it's supposed to be for strings...), it's just a beginning.... We need to wait for someone more experienced to arrive. http://sylvain.sab.free.fr/autres/ccmsdz/trouver_fichiers/  But anyway, the script doesn't work, we need to look at the source: http://sylvain.sab.free.fr/autres/ccmsdz/trouver_fichiers/?source -- Sylvain

Powo2 · Answer

Alright, thank you very much, that's really nice of you ^^ I did a lot of research before coming here, so if it can help you, I found these two links that seemed interesting but that I don't understand or that I can't exploit  https://codes-sources.commentcamarche.net/  https://codes-sources.commentcamarche.net/  And also, is it not too much to ask how to use a script :$ ?

Ssylvainsab · Answer

To use a script, go to the corresponding address. Since the script is in PHP, it can only run on a server. Check the source and copy the file to your site if you have one. http://sylvain.sab.free.fr/autres/ccmsdz/trouver_fichiers/?source -- Sylvain

Ssylvainsab · Answer

But the problem is that we don't know the file name, there are several variables.  What we need to do is find a way to list all possible urls (as you mentioned), but I don't know how to do that. Then, we need to check each url and display it if it's valid. But it's complicated. (for me). I think we should be able to do this with an array and some foreach, plus a few regex and file_exists...  Otherwise, in another language, but I only know PHP as a programming language.  -- Sylvain

Powo2 · Answer

I don't have a website, and when I click on the link, it takes me to an error page.

Ssylvainsab · Answer

I know, it's normal since the script is invalid, it's just a draft. We need someone with more experience to help us. -- Sylvain

Powo2 · Answer

So nobody else is capable of helping me solve this problem?

Ssylvainsab · Answer

In a little while, if no one has responded to your topic, you can make another post; I don't think it will bother anyone.  Especially since we've made some progress on the issue.  However, I did say in a little while.  See you later -- Sylvain

lami20j · Answer

Hello,   I first need a way to create a list of URLs.  If I understood correctly from your message changes, a part of a URL needs to be found by combining a certain number of alphanumeric characters.  If we have a string of 3 characters, for example xyz, then we have a total of 3! combinations, which gives us 6 combinations. That's a simple case. Adding the layer of the alphabet with its 26 letters makes the number of possibilities really large.  And if we add another layer, the number of characters to combine increases, so keep in mind that 6! already gives us 720 combinations.   For example, to create a list of URLs, you can use a simple script. For the URL http://feuks.free.fr/colles/colles_physio/P106xxx.rar  and assuming xxx is a combination of the letters d,a,c, you can have something like this in Perl (of course, you can use PHP, Python, Perl, or another scripting language)@combi = qw( dac, dca, acd, adc, cad, cda); $url = 'http://feuks.free.fr/colles/colles_physio/P106xxx.rar '; foreach (@combi) { (my $res = $url) =~ s#(http://.*/P106)(?:\w{3})(\.rar)#$1$_$2#; print "$res
"; }In this case, I wrote the combinations by hand; 6 is manageable, but starting from 4, it's better to write a script to permute the characters.   And then a way to test them to see which one is correct.   Once the list is created, testing it can be simple or complicated. If you only have 6 links to test, you just need to check them one by one :-)) And that's already boring.  If you have several, personally, I would write a script for that. Maybe there is software that can test a list of URLs for you; I don't know.  lami20j

Ssylvainsab · Answer

Don't you have a way to create a list with the 3 variables? I think I can manage the verification. The 3 variables? What do you mean?   Thank you for your help lami20j, but he doesn't know much about programming and neither do I.  What we need to do is a list of possible URLs. To check the links, it's very simple.  So if you have an idea of the script that corresponds, you can do it.  Maybe it needs a loop, or an array and a foreach. I'm speaking in PHP because it's the only programming language I know (a little).   Maybe for that he can have a software that tests a list of URLs, I don't know. Yes, there is Xenu Link's Sleuth.  See you later -- Sylvain

Powo2 · Answer

Thank you very much for the software, it's exactly what I needed to check.  Now it "just" (lol ^^) needs to create a list that gathers all the possibilities...  The 3 variables being the 3 x's in the addresses (they can be different, identical but they are always 3 letters)  http://feuks.free.fr/colles/colles_anat/anat1xxx.rar http://feuks.free.fr/colles/colles_bio/b110xxx.rar http://feuks.free.fr/colles/colles_bioch/B104xxx.rar http://feuks.free.fr/colles/colles_biophys/b106xxx.rar  I would like a list of this type with all the possibilities  http://feuks.free.fr/colles/colles_bioch/B104aaa.rar http://feuks.free.fr/colles/colles_bioch/B104aab.rar http://feuks.free.fr/colles/colles_bioch/B104aac.rar ... http://feuks.free.fr/colles/colles_bioch/B104frt.rar ... http://feuks.free.fr/colles/colles_bioch/B104zzz.rar

lami20j · Answer

Re,  Attention, the situation is getting complicated.  Up to colles_ it's fine but then everything changes, not just the xxx  In short, you have the basis  http://feuks.free.fr/colles/colles_ and then your combinations.  How do you get that?  http://feuks.free.fr/colles/colles_anat/anat1 http://feuks.free.fr/colles/colles_bio/b110 http://feuks.free.fr/colles/colles_bioch/B104 http://feuks.free.fr/colles/colles_biophys/b106  lami20j

Powo2 · Answer

After glues_ these are data that I know so I could modify them... it's just the 3 letters x that are unknowns.

Powo2 · Answer

After glues_ these are data that I know so I could modify them... it’s just the 3 letters x that are unknowns.

Ssylvainsab · Answer

After colles_, these are data that I know so I could modify them... Yes, but then he will have to give you the list for each thing you know after colles_, then. If you don’t have many, you could change them yourself, but if you have a lot...  Once we make the list for one of the possible URLs, it will simply be a matter of doing a search/replace.  For example, let's make the list of all the possible URLs for http://feuks.free.fr/colles/colles_bioch/  We only make this list, we check the links with Xenu, then we do the search and replace: We search for bioch And replace it with biophys Which gives http://feuks.free.fr/colles/colles_biophys/ Then we recheck. Then we do search and replace again, but this time for "bio"  etc...  and like that for each of your possible URLs. That will be up to you to do.  But why do you want to do that? -- Sylvain

lami20j · Answer

Hello,  Here is the script#!/usr/bin/perl use strict;use warnings; my $url='http://feuks.free.fr/colles_bioch/B104xxx.rar'; foreach ( 'aaa'..'zzz' ){ (my $r = $url) =~ s#(http://.*B104)\w{3}(\.rar)#$1$_$2#; print "$r
"; }lami20j  P.S you get 17576 URLs

Ssylvainsab · Answer

Yes, but is that the Perl script?  I'll give it a try.   EDIT: There it is, I have a file with the .pl extension But how do I execute it? -- Sylvain

Ssylvainsab · Answer

Oula, I don't have much knowledge in PHP, so it's going to be hard to adapt it...  I'm already going to put the script code on my site. http://sylvain.sab.free.fr/autres/ccmsdz/trouver_fichiers/ http://sylvain.sab.free.fr/autres/ccmsdz/trouver_fichiers/?source -- Sylvain

lami20j · Answer

Re,  otherwise I’ll give you the list that I got ;)   lami20j

Ssylvainsab · Answer

If you could make a file for me, or give me the list, I would put it on my site and check the links. -- Sylvain

Powo2 · Answer

Lol, I just dropped out ^^ I went to eat, but yes, there aren't many files, there's bio, biophy, bioch, anat, physio and that's it... after that, in each folder, there are a certain number of files starting with b103, b104, b105... or p1031, p104 and so on, depending on the file... but I could change the beginning of the address manually...  So, how should I execute the script then?

lami20j · Answer

Re,  Powo2, I suppose you are also on Windows. So you need to install Perl.  On Linux  perl script.pl > result  For now, I will take your options and bio...  and 103 I will create the files, and then you'll be able to manage. More than half of the work will be done.  Is that okay with you?  lami20j

Powo2 · Answer

The 5.9.2 version of Perl? I'm having a hard time with it, I can't even install it...  I'm a beginner :) could you explain how to run the script with version 5.9.2 that I downloaded?  Do I need ActivePerl?

lami20j · Answer

Re,  ok, I'll let you know but first I need to finish the script so that I don't have to run it multiple times while manually changing the parameters.  So what I need to know is  b103 b 104, b105... or p1031 p104 and so on let's be more precise please.  lami20j

Powo2 · Answer

Okay wait, I'll give you all the addresses but there are about a hundred of them, that's why I didn't want to overdo it  colles_anat/anat1xxx.rar colles_anat/anat2xxx.rar colles_anat/anat3xxx.rar ... colles_anat/anat18xxx.rar  colles_physio/P101xxx.rar colles_physio/P102xxx.rar colles_physio/P103xxx.rar ... colles_physio/P113xxx.rar  colles_bio/b103xxx.rar colles_bio/b104xxx.rar colles_bio/b105xxx.rar ... colles_bio/b119xxx.rar  colles_biophys/b101xxx.rar colles_biophys/b102xxx.rar colles_biophys/b103xxx.rar ... colles_biophys/b136xxx.rar  colles_bioch/B101xxx.rar colles_bioch/B102xxx.rar colles_bioch/B103xxx.rar ... colles_bioch/B119xxx.rar

lami20j · Answer

Re,  for your hundreds, the processing won't take that long.  lami

Powo2 · Answer

Here it is, that's why changing the base of the address each time is a tedious job, I wanted to do it myself... I gave you all the changes above, there you go :)

Powo2 · Answer

It's not working? Because otherwise I can very well do it manually, I don't mind.

lami20j · Answer

Re,  Sure, that works. But I was a bit busy ;)  Here is the script. I tested it at home on Linux. I get a file of 81 MB with 1,810,328 lines. Execution time 2 to 3 seconds#!/usr/bin/perl use strict;use warnings; my @prefix; push @prefix, "anat/anat$_" for 1..18; push @prefix, "physio/P$_" for 101..113; push @prefix, "bio/b$_" for 103..119; push @prefix, "biophys/b$_" for 101..136; push @prefix, "bioch/B$_" for 101..119; my $url_base = 'http://feuks.free.fr/colles_'; map { $_ = $url_base.$_ } @prefix; foreach ( @prefix ) { foreach my $l ( 'aaa'..'zzz' ) { print "$_$l.rar
"; } }lami20j

Powo2 · Answer

Ah :) so could you tell me how to use this script? which software... even which button to press ^^ because I'm more than a beginner ^^ I downloaded ActivePerl 5.8 and I have Windows XP SP2

Powo2 · Answer

Ah, I managed to do it; it opens a command prompt window and displays all the addresses in about 2 minutes, but then it disappears. Is there a way to save them to a text file?

lami20j · Answer

Ah yes  you need to make a redirect.  When you execute it in the MsDOS console, write at the end  > C:\url.txt  The result will be in the file url.txt in C:\  Note that the duration of 2 minutes is due to the display on the screen with the redirect it will be faster.  lami20j

Powo2 · Answer

No, but the window disappears by itself and I have no interaction. Can't you add a line to the program to write all the responses in a notepad?

lami20j · Answer

Yes, I can. Wait   But how do you execute it? Do you use the mouse?

Powo2 · Answer

Okay, thank you very much, I'm waiting :)  I created a PL file that is executable with ActivePerl and I copied your script into it. And I double-click on it to run it, yes.

lami20j · Answer

Look here on how to run the script.  You need to open the DOS window (look in Start/Programs/Accessories/)  https://www.cjoint.com/?kwpnGNIIPv   So I assume you have installed Perl in C:\  I named the script url3.pl  The result will be in the file resultat.txt located in C:\   The execution is quick  however, opening the file will take longer (due to its size)   lami20j

Powo2 · Answer

Oooh :) thank you very much, it works... I'm going to try with Xenu now and see if I find it, I'll keep you updated... it's really nice, thank you.

Ssylvainsab · Answer

Well, let's say I was useless in the end, but it's still good! Your problem is (finally) resolved Powo2?  I'm checking the resolved button.  See you later -- Sylvain PS: Thanks for your help Lami20j!!

Ssylvainsab · Answer

Well, let's say I was useless for the end, but it's good anyway! But no, you shouldn't say that. You did half the work :-) Thank you. I was talking about the end. And anyway, I was joking. :D By the way, you still haven't answered my question Powo. Why do you want to do this? See you later -- Sylvain

Powo2 · Answer

Thank you very much, it works, but it might take some time, and I told you in my post 12 why I wanted to do this ^^  I'm going to confess to you, these are paid practice quizzes (in medicine)... and I don't have enough money to buy them all... after paying for a few, I noticed a redundancy... so I guess these are addresses of this type with random letters.  Know that these kinds of people failed medicine and sell the things they got for free; it's very useful, but they're trying to make money off the students. I'm going to be able to benefit quite a few people, and this year will be a little less about money and much more about work.  In any case, thank you very much to both of you...

Powo2 · Answer

As I'm asked to summarize:   I first use the following script with Perl:   #!/usr/bin/perl  use strict; use warnings;   my @prefix;   push @prefix, "anat/anat$_" for 1..18;  push @prefix, "physio/P$_" for 101..113;  push @prefix, "bio/b$_" for 103..119;  push @prefix, "biophys/b$_" for 101..136;  push @prefix, "bioch/B$_" for 101..119;   my $url_base = 'http://feuks.free.fr/colles_';  map { $_ = $url_base.$_ } @prefix;   foreach ( @prefix ) {  foreach my $l ( 'aaa'..'zzz' ) {  print "$_$l.rar
";  } }   Then to find out which addresses are valid, I use the Xenu software at: http://sylvain.sab.free.fr/logiciels/xenu.php

[URL] Found a URL with missing part

54 answers

Asus tuf gaming b650-plus wifi dram issue

My computer stays connected for very little time

Mouse, fullscreen mode

Toshiba c70d-a-111 image/recovery disk search

Problem with open office spreadsheet

Which smart tv to choose?

No internet connection possible

How to delete c:\program files(x86)\common files\autodesk

Unable to open xsl and xslx files

Honor x 5c how to unlock the n° in the sytanager