grep | awk

[pne]

Occasionally I see people writing little pipelines using both 'grep' and 'awk'... and it makes me wonder why you start two processes when one will do. (A bit like those who start pipelines with cat file | ... where <file ... will do just as well, and save one process.)

For example, why not replace ... | grep blurfle | awk '{ print $2 }' with ... | awk '/blurfle/ { print $2}'? Or replace ls -l | grep -v '^d' | awk '{total+=$5}END{print total}' with ls -l | awk '$1 !~ /^d/ {total+=$5} END {print total}'?

Especially since most of those recipes use a very simple argument to grep—typically a fixed string, or -v plus a fixed string—which would seem trivially convertible to part of an awk script even without having to learn the full awk language.

Comments

[setone.livejournal.com]

I dunno if you like stealing LJ icons and not giving credit where it is due but the icon entitled- more and more is ACTUALLY from a friends account. Give her credit where it is due!
Her name is: takichan

Another thing is you even gave the icon the same name as she did.

Thanks

'more and more' icon

[pne.livejournal.com]

I've seen this icon on a couple of people's accounts; the one I gacked it from did not show any "created by" so I didn't know whom to credit. I think it was someone's default icon and didn't have any keyword at all, so naming it something that's nearly the same is coincidence.

the icon entitled- more and more is ACTUALLY from a friends account.

I see that the icon, or a very similar one, is indeed one of takichan's. However, that doesn't mean that she made it; she could have got it from someone else.

If she did take the original pictures / shoot the original video and converted the pictures or screenshots to an animated icon, have her contact me herself and tell me.

Re: 'more and more' icon

[setone.livejournal.com]

I've already seen that you've replied to one of her enteries. But besides that alot of icons that end up on LJ aren't shot by that person so does that mean if she didn't fully create it you won't give her credit?
The icon wasn't on LJ before she brought it over because it came from a friend who knew the orginal creator of the icon. She simply got permission to use it. Why she didn't credit him is because he didn't care about crediting.

Yes she'd like credit where it is due..But moreso she wants people to ask permission instead of stealing/taking it.

Re: 'more and more' icon

[pne.livejournal.com]

alot of icons that end up on LJ aren't shot by that person so does that mean if she didn't fully create it you won't give her credit?

I'd take the amount of work and level of originality into consideration, at least.

For example, if someone takes a picture and merely crops it and adds a 1px black border, I don't think that's sufficient work to warrant crediting (unless the cropping or compositing was nontrivial). If someone takes three images from a web comic and animates them nicely, then they may warrant credit since animation is not something that trivial.

You mention "credit where it is due"... that was kind of what I was thinking. If you do something, then you deserve a certain measure of recognition for it, roughly proportional to the amount of work you put into it. If you do little, then the amount of recognition you can expect will be small; if you do a lot, then it will be bigger.

The icon wasn't on LJ before she brought it over because it came from a friend who knew the orginal creator of the icon. She simply got permission to use it.

So it would make sense to credit the creator in this case, if all she did was introduce it to LiveJournal.

(If merely bringing an icon to someone's attention deserves credit, then someone gacking it from another gacker would have to credit with "gacked from bla who gacked it from bla who gacked it from bla who added a 1px black border to an icon of bla's who cropped and resized it from a picture of bla's.)

Why she didn't credit him is because he didn't care about crediting.

Though if he did not expect credit, that particular issue is moot.

Yes she'd like credit where it is due

This is one reason why I'm trying to determine how much, if any, credit is due to whom.

I'm not against giving credit where it is due; however, I don't think that just making something into an icon is enough to merit credit. (For example, if someone grabbed my 'bus' or 'peony' icons, which I made, I wouldn't expect credit, since all I did was resize them and introduced them into LiveJournal. If someone wanted to credit me, I'd be fine with that, but I don't think I could expect it.)

But moreso she wants people to ask permission instead of stealing/taking it.

That seems fair enough.

Though "stealing" doesn't sound like the right word if it's not her icon; you said that "it came from a friend", and even that friend is not the original creator, so I'm not sure how it can be considered to belong to takichan or her friend.

I agree that it's polite to ask before you use an icon that someone else is using; it's the wording that I find not appropriate.

Re: 'more and more' icon

[nameandnature]

See also stop_i_bitches.

I tend to use awk that way because it's so strongly associated with selecting fields for me, rather being a general purpose tool.

[nakedcelt.livejournal.com]

I like cheese.

[ewx.livejournal.com]

I suspect there are two main ways in which these arise. Firstly not everyone actually knows anything like the full feature set of the tools they use; they just know that grep selects line and that awk selects space-separated fields; but not that awk is capable of doing both operations. So if you want a particular field from a subset of lines then commposing grep and awk is the obvious answer.

Secondly a common strategy when constructing a pipeline is to do so one command at a time, and review the output before adding the next one. With that approach it's both less typing and also less error-prone to add a new entry than to go back and replace the last one.

[pne.livejournal.com]

Good point on the second one.

In principle one could say ... | awk '/foo/' when testing and add an action later, but I suspect that not many people use awk simply for printing out all lines that match a pattern.

[crschmidt.livejournal.com]

I definitely fall into the first. I know nothing about awk, and some about grep. This is similar to something pne mentioned earlier to me at one poinnt: why did I use seperate sed -e processes in a pipeline, rather than just putting them all in one process? The truth of the matter was just that I didn't know the latter was possible.

[ataltane.livejournal.com]

I think both reasons go some way towards explaining why I construct complex lines in the way that Philip disapproves of. However, I have two minor things to add:

First, I care not a whit for the number of processes invoked. I want to get the job done, and I don't want to spend time optimising the command line (this only makes sense in combination with the reasons why I positively like to construct command lines piece-meal).

Second, as someone who regularly uses the bizarrely wasteful form 'cat file | quz | baz', I can only say in my defence that I like it. I mean, the meaning appeals to me somehow :).

[ataltane.livejournal.com]

^fence^fense

defen[cs]e

[pne.livejournal.com]

'defence' is fine as a UK spelling; just claim to be from there :)

Re: defen[cs]e

[ataltane.livejournal.com]

Hmm, yes. God damn this increasing illiteracy. I used to be able to spell perfectly, and I probably gave unwanted advice to other people. So I probably deserve it :).

Anyway, whatever about the UK (where's that? the Ukraine? :), 'defence' is a perfectly fine Irish spelling. I occasionally use "European English" to include the Irish variety. This has the disadvantage that it sounds absurd, but the usual "British English" sounds absurd anyway, to the millions of English users outside of Britain but within ... these isles.

Re: defen[cs]e

[ataltane.livejournal.com]

Clarification: your spelling advice wasn't unwanted. No such implication intended!

[ewx.livejournal.com]

The initial redundant 'cat' I find harder to explain; it's more typing than using stdin redirection.

[ataltane.livejournal.com]

It's a little bit more typing; less than the four labouriously depressed keys that might be imagined; rather a quick gesture of the left hand. Whatever pleasure the idea of beginning a sequence of commands with a "cat foo |" brings me, I judge it worth the slight additional effort. :)

[catamorphism.livejournal.com]

This is an interesting thread. People write things like "cat foo | awk '{print $1}' | blah blah blah" because when you think about the steps in the pipeline, you think about the step consisting of outputting the entire file as coming first. "awk '{print $1}' < foo" doesn't seem as natural because the second step (selecting the first field) comes first. It would be useful if the shell had a syntactic form like "< foo awk '{print $1}'" that would mean the same thing as "cat foo | awk '{print $1}", without starting an extra process.

As for not writing "awk '/blurfle/ {print $2}', it's the same sort of thing. One likes to think of the process of selecting lines with the string "blurfle" and then selecting the second column as being separate steps, and separating them out by constructing a pipeline makes that clear. It's interesting to think about what would happen if the shell were much smarter and could optimize scripts like "grep blurfle | awk '{print $2}'" into "awk '/blurfle/ {print $2}". This would be an idea somewhat akin to deforestation, which is what my research was on.

In either case, it's an illustration of why syntax matters: people will write less efficient programs in order to make the meaning of the program more understandable. It's nice if they have syntax that allows them to write understandable, efficient programs (in the first case) or back-end tools that turn the undertstandable programs into efficient programs (in the second case).

[pne.livejournal.com]

It would be useful if the shell had a syntactic form like "< foo awk '{print $1}'" that would mean the same thing as "cat foo | awk '{print $1}", without starting an extra process.

Uh, TTBOMK "< foo blabla" is the same as "blabla < foo"; i.e. you can put the redirection at the beginning or the end and either way will work.

[noidd.livejournal.com]

cat filename | .... is more intuitive. Stuff in a chain of piped commands go from left to right... that syntax is more readable imho...

command1 < filename | command2 | command3 is less intuitive to me than:
cat filename | command1 | command2 | command3

Hell, it was only a year ago I broke the habit of doing:

cat filename | grep expression

ymmv