Select next line CSH foreach

Bon, c'est pour un shell bash, il te faudra adapter la syntaxe pour le csh...

$ cat plop  simon ok x febgeg simon ok x rhedg simon ok x erze simon ok x srg e fabien ok x nrteth fabien ok x tehhet seb ok x et ee yoann ok x eth yoann ok x et he yoann ok x ehe yoann ok x egr yoann ok x ereh yoann ok x ete $ cat foo.sh  #! /bin/csh #set -xv while ( $line = <>) sed -i "s/${line% *}/ok ${line#*}/" plop end < <(awk '{ print $1 }' plop | uniq -c | awk '{ print $2,$1 }') $ ./foo.sh  $ cat plop  simon ok 4 febgeg simon ok 4 rhedg simon ok 4 erze simon ok 4 srg e fabien ok 2 nrteth fabien ok 2 tehhet seb ok 1 et ee yoann ok 6 eth yoann ok 6 et he yoann ok 6 ehe yoann ok 6 egr yoann ok 6 ereh yoann ok 6 ete $ 

;-))

--
Zen my nuggets ;-)
Faites un geste pour l'environnement, fermez vos fenêtres et adoptez un manchot.

$ cat plop  simon ok x febgeg simon ok x rhedg simon ok x erze simon ok x srg e fabien ok x nrteth fabien ok x tehhet seb ok x et ee yoann ok x eth yoann ok x et he yoann ok x ehe yoann ok x egr yoann ok x ereh yoann ok x ete $ cat csh_foo.csh #! /bin/csh foreach line ( 'awk '{ print $1 }' plop | uniq -c | awk '{ printf "%s|%s\n",$2,$1 }'' ) set line = "$line:gas/|/ /" set argv = ( $line ) sed "/$1/{s/ok x/ok $2/}" plop > blop mv blop plop end $ ./csh_foo.csh $ cat plop simon ok 4 febgeg simon ok 4 rhedg simon ok 4 erze simon ok 4 srg e fabien ok 2 nrteth fabien ok 2 tehhet seb ok 1 et ee yoann ok 6 eth yoann ok 6 et he yoann ok 6 ehe yoann ok 6 egr yoann ok 6 ereh yoann ok 6 ete $

;-))

--
Zen my nuggets ;-) Do something for the environment, close your windows and adopt a penguin.

Hello,

In case you didn't know... there are ready-to-use tools available on GNU/Linux...

$ cat plop simon ok x simon ok x simon ok x simon ok x fabien ok x fabien ok x seb ok x yoann ok x yoann ok x yoann ok x yoann ok x yoann ok x yoann ok x $ uniq -c plop  4 simon ok x 2 fabien ok x 1 seb ok x 6 yoann ok x $

;-))

--
Zen my nuggets ;-)
Do something for the environment, close your windows and adopt a penguin.

Re,

Well, there must be something simpler, but I'm thinking of something like this:
- we get the number of occurrences and store them in a temp file

:~$ perl -ane '$h{$F[0]}++;END{print "$_:$h{$_}\n" for keys %h}' visiteurr > visiteurr.occ lami20j@debian-acer:~$ cat visiteurr.occ seb:1 yoann:6 simon:4 fabien:2

- we use the temp file and insert the number of occurrences into the file

:~$ cat visiteurr simon ok x febgeg simon ok x rhedg simon ok x erze simon ok x srg e fabien ok x nrteth fabien ok x tehhet seb ok x et ee yoann ok x eth yoann ok x et he yoann ok x ehe yoann ok x egr yoann ok x ereh yoann ok x ete :~$ perl -ne '$h{$1}=$2 if /(.*):(.*)/;s/^(.*?)\s(.*?)\sx(.*)/$1 $2 $h{$1} $3/ and print' visiteurr.occ visiteurr simon ok 4 febgeg simon ok 4 rhedg simon ok 4 erze simon ok 4 srg e fabien ok 2 nrteth fabien ok 2 tehhet seb ok 1 et ee yoann ok 6 eth yoann ok 6 et he yoann ok 6 ehe yoann ok 6 egr yoann ok 6 ereh yoann ok 6 ete 

--
GNU/Linux: Linux is Not Ubuntu! Choosing which Linux to use does not mean your favorite Distribution,
106485010510997108

Hello,

1st command

perl -ane '$h{$F[0]}++;END{print "$_:$h{$_}\n" for keys %h}' visiteurr > visiteurr.occ

The role of this command is to count the number of occurrences of the word at the beginning of the lines in the file.

For this, I use a data structure called hash or associative array.
This data structure allows accessing array elements by a key (which is a string).
Each key corresponds to a value (which can be a string, a number, an array, a hash, a reference, basically anything ;-)
This results in the following presentation

%hash = ( "key1" => "value", "key2" => "another value", .... "keyN" => "and yet another value", ); 

Note that the key is unique.

In your example, the command will go through each line of the file.
Since we are looking for the number of occurrences of the first word of each line, we just need to consider the first word as the key, and since it should be unique, I will just count the value afterwards.

Here’s what happens under the hood.

Processing the first line
the key is simon and the value will be 1

Processing the second line
the key is simon and the value will be 2 (the value is incremented with each occurrence)

All this for all the simon, regardless of the line number in the file (so the lines starting with simon don’t need to be grouped)

When we reach fabien, that’s a new key, and similarly to the simon key, the value will be incremented and so on until the last line of the file.

In the end, the hash looks something like this (note that we can sort the hash but not needed in this case) which is internal and thus random and not in the order of creation of the hash

 %h = ( "seb" => 1, "yoann" => 6, "simon" => 4, "fabien" => 2, );

At this point, the hash is in memory and needs to be saved somewhere; I chose a file.
The block END{} ensures that once it reaches the end of the file, the hash is displayed.
To write to the file, I used simple redirection of STDOUT (standard output, the screen) to a file.

That’s how the 1st command works.
The options used allow splitting the words of each line into an array @F and then I use $F[0] - the 1st element (simon, seb, fabien, yoann)

The 2nd command

perl -ne '$h{$1}=$2 if /(.*):(.*)/;s/^(.*?)\s(.*?)\sx(.*)/$1 $2 $h{$1} $3/ and print' visiteurr.occ visiteurr

This command reads both files:
- the one created by the 1st command which contains the number of occurrences
- the original file.

The command consists of two lines of code separated by a semicolon
$h{$1}=$2 if /(.*):(.*)/
and
s/^(.*?)\s(.*?)\sx(.*)/$1 $2 $h{$1} $3/ and print

The command $h{$1}=$2 if /(.*):(.*)/ at the moment of reading the 1st file will recreate the hash.
This time the separator is no longer a space but a colon
(.*):(.*) it is a regular expression that could be translated like this

. means any character
* is a quantifier that allows finding 0, 1, or any number of characters
() the parentheses are for capturing the found pattern
: is the literal character

The captures are numbered from 1 to .... and the corresponding variables are $1, $2 .....

What’s interesting is that the hash will be filled only if the line contains a : (this could pose memory problems with no results if the original file contains :)
We could improve by using start and end string anchors. (^ - start; $ - end)

You might wonder why we didn’t do it all at once instead of creating a temporary file.
If the file is large (let’s say millions of lines) then just imagine how much RAM + swap we would need to store all that.
Well, the worst case would be if the original file contained one key per line, but in that case it would not be necessary to count the number of occurrences, and in such a case adding 1 in the column would suffice

So $h{$1}=$2 if /(.*):(.*)/ briefly says: fill the hash with key => value only if the line read from the file contains :

At the end of reading the 1st file, the hash is filled and the reading of the original file begins.

s/^(.*?)\s(.*?)\sx(.*)/$1 $2 $h{$1} $3/ and print

Knowing that the separator is the space, it is sufficient to split the words and then replace the x with the corresponding value found in the hash

s/MOTIF/REPLACEMENT/ is the substitute function that allows replacing the left side with what is on the right

The MOTIF part
s/^(.*?)\s(.*?)\sx(.*)/

s/
^ - start anchor
( - start 1st capture - $1
.*? - any character 0, 1 or any number of times but avoid greediness
) - end 1st capture
\s - looks for a space
( - start 2nd capture - $2
.*? - any character 0, 1 or any number of times but avoid greediness
) - end 2nd capture
\sx - is the field concerned for the change
( - the 3rd capture - $3
.* - any character 0, 1 or any number of times, greedy this time
) - end of the 3rd capture

Be careful, if the modified column does not contain x then the regex should be changed

The REPLACEMENT part

/$1 $2 $h{$1} $3/ and print

/
$1 - the 1st capture
$2 - space the 2nd capture
$h{$1} - space and see (number of occurrences)
$3 - space and the 3rd capture
/ and print - end of replacement and display

number of occurrences $h{$1}
The 1st capture is the first word of the line.
$h{$1} for example, when the word is simon we have:

$h{"simon"} and in the hash we saw that the value of simon is the number of occurrences found by the 1st command, so 4

This substitution is applied for each line.

There you go, I hope it’s a bit clearer.

He is the Perl specialist ;-))
Not a liar regarding the connection, but for the rest yes ;-))

Hi,

To be honest, my test is based on an example that doesn't seem to match your file.
For that, I might need your file.
Can you send it to me by email?

One small clarification ... what's the difference between ".*?" and just ".*"?

Here's an example to see the difference.
You notice that when I use .* a, $1 is xigenc - .* has consumed everything up to the last e, so the longest string.
On the other hand, when I use .*?, then $1 is xig - .*? has consumed up to the 1st e, so the minimal string.

 :~$ echo exigence exigence :~$ echo exigence | perl -ne '/e(.*)e/ ; print "$1\n"' xigenc r:~$ echo exigence | perl -ne '/e(.*?)e/ ; print "$1\n"' xig

Hi,

For that, I might need your file.
Can you send it to me by email?
Already asked, but that's not possible, however the original lines look like this;-\

Hello,

Already asked, but it's not possible, however the original lines are similar

Well, that's exactly what bothers me that it gathers, but no one mentions what lies behind the non-printable characters (space, tab, or I don't know what else ;-)

I will try to generalize.

Gotta deal with it... but that's when you really see the true beasts in the end ;-))

Re,

I don't understand how to "simulate" columns...

Do you know?

So let's try to figure out the structure of your file.
With this command, all characters other than spaces and tabs are replaced by A and the others by their ASCII code.

perl -ne 'while(/(.)/g){my $x=$1;($x=~/\s/)?(print " ", ord($x), " "):print "A"};print "\n"' visiteurr > visit.struct

Then you put the file visit.struct on cjoint.com

As proof, here's what it displays on my end

~$ cat visiteurr simon ok x febgeg simon ok x rhedg simon ok x erze simon ok x srg e fabien ok x nrteth fabien ok x tehhet seb ok x et ee yoann ok x eth yoann ok x et he yoann ok x ehe yoann ok x egr yoann ok x ereh yoann ok x ete ~$ perl -ne 'while(/(.)/g){my $x=$1;($x=~/\s/)?(print " ", ord($x), " "):print "A"};print "\n"' visiteurr > visit.struct lami20j@debian-acer:~$ cat visit.struct AAAAA 32 AA 32 A 32 AAAAAA AAAAA 32 AA 32 A 32 AAAAA AAAAA 32 AA 32 A 32 AAAA AAAAA 32 AA 32 A 32 AAA 32 A AAAAAA 32 AA 32 A 32 AAAAAA AAAAAA 32 AA 32 A 32 AAAAAA AAA 32 AA 32 A 32 AA 32 AA AAAAA 32 AA 32 A 32 AAA AAAAA 32 AA 32 A 32 AA 32 AA AAAAA 32 AA 32 A 32 AAA AAAAA 32 AA 32 A 32 AAAA AAAAA 32 AA 32 A 32 AAA 32 

--
GNU/Linux: Linux is Not Ubuntu! Choosing a Linux doesn't mean your favorite Distribution,
106485010510997108

RE ^^ so after having extensively modified the code to try to adapt it ^^ here's what I got:

perl -ne '$h{$1}=$2 if /(.*):(.*)/;s/^(.*?)\tmodification\t(.*)/$1\t$h{$1}\t$2/ and print' texte.txt.occ texte.tmp.3_2 > texte.txt.tmp.3_3

with lines of this type:

simon modification 9999.00 test 999.00 tes2 test3 pierre 99.00 test4 yoann 99.00 99.00 grande_phrase 9999999.00 99.00 99.00 9.00 99.00 didier

and it works great

Unfortunately, I am not allowed to output even a modified document from my company ... Additionally, the file has 16,000 very, very long lines ^^ you might say it's not a big deal lol I have one that has over 4 million lines =) (more than 200Mo).
In my example above, each word is in a different column and the result should be:

simon 8 9999.00 test 999.00 tes2 test3 pierre 99.00 test4 yoann 99.00 99.00 grande_phrase 9999999.00 99.00 99.00 9.00 99.00 didier

if Simon appears 8 times in the first position.

What do you think of my modifications? Is it normal that it works or is it a stroke of luck and it won't work all the time?

Thanks guys, that's really nice