Extract lines from file, add to new files

[avi.e.gross at gmail.com]

This discussion has circled back to where it started. It illustrates quite a
few points about how many different ways someone can do something as well as
doing it using different tools and also about how others may see aspects of
mission creep as they look for making it perfect when it need not be. I mean
the OP wants a simple personal tool and is not currently concerned with lots
of things.

The data seems to be in a data base and can be extracted from there into one
or two files mostly for convenience. Presumably, all the usual
additions/deletions/modifications are being done in the DB. If so, then as
long as his latest vision of having two synchronized files is done
consistently by making both, they are in sync.

In addition, it sounds like the original idea was a tad too simple. The OP
wanted to add a single-line or so at the head of a message that simply
interpolates a name. This could be handled with all kinds of
form-letter-merge methods as well as search and replace methods of great
generality. But, for now, the OP is asking for something truly simple and
would be happy with a solution that is slow and clunky and uses a few
utilities in a shell script on LINUX. I took a look at doing this on bash
and ran into some surprises as this would have been trivial for me in a
shell like ksh. Setting variables is now done in a subshell which was done
why? All I know is if you could simply read a file one line at a time in a
loop and set one or more variables, then it becomes easy to use something
like combining an echo and a cat of the original message to make a message
with a variable heading and feed that to the mailer. And if the storage
format was CSV, again, you could easily peel it apart using something like
cut or awk or whatever.

The main reason for trying it in python may have been frustration with doing
it as a shell script. Fair enough. But the idea of two files may have been
made in the first place by the original idea of such a script. The question
posed was sort of how to search in one and then somehow find the
corresponding part of another. The mindset of the OP may have been focused
in a direction we were not clear on. As an example, there are ways to search
file A and get back a line number of the first match and then using perhaps
another utility you ask it to scan file B and just print out that line
number and nothing else.

The current solution the OP is considering sounds like a loop that goes
through both files in parallel. This would mean when one loop finds what it
is looking for, the other delivers what is in the same location. That may
work well enough for the purpose.

But I have learned over the years that having a few well-designed tools and
ideas and using them consistently, is often a good way to get results even
if it is not as imaginative and efficient as some other. If you are allowed
to design something from top to bottom, there may be very nice solutions.
But if asked to just do one small part in a design that already feels it has
solved other aspects, then suggesting they start over is often not taken
well. LOL!

I happen to think some storage format such as CSV would be a good idea IFF
doing this in Python for some reasons I mentioned earlier. I am not in love
with CSV and use other data formats as needed and some others would be fine
as long as your toolkit included some easy and reliable way to read the
contents in and manipulate to get your results. I do programming in multiple
languages and whether I am using python or something like R, I often end up
importing the data into some data.frame format, performing a pipeline of
manipulations using it and perhaps other such structures and generating
results and optionally saving some back to files. Tons of well-tested and
comfortable tools can be used if the data is set up right so even when it
can be easily done by hand, it may not be worth the effort as the standard
tools and tricks work and work fast enough.

Rich (AKA the OP) seems satisfied with having a solution. It may not be
ideal but for his needs, ...


-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=gmail.com at python.org> On
Behalf Of Thomas Passin via Python-list
Sent: Wednesday, January 31, 2024 10:36 AM
To: python-list at python.org
Subject: Re: Extract lines from file, add to new files

On 1/31/2024 9:05 AM, Rich Shepard via Python-list wrote:
> On Tue, 30 Jan 2024, Thomas Passin via Python-list wrote:
> 
>> If I had a script that's been working for 30 years, I'd probably just use
>> Python to do the personalizing and let the rest of the bash script do the
>> rest, like it always has. The Python program would pipe or send the
>> personalized messages to the rest of the bash program. Something in that
>> ballpark, anyway.
> 
> Thomas,
> 
> A bash shell script looks easier for me and more promising. Using a while
> loop (one for the name file the other for the address file), and sed for
> putting the name at the head of the message replacing a generic
placeholder
> should work with the existing for loop script.

Sounds good.  I'd still be a bit worried about the two files getting out 
of sync, as others have mentioned.

-- 
https://mail.python.org/mailman/listinfo/python-list