[rfc] sed option `-i' (edit in place)

[Cristian Ionescu-Idbohrn]

On Sun, 26 Aug 2007, Denys Vlasenko wrote:

> On Sunday 26 August 2007 16:00, Cristian Ionescu-Idbohrn wrote:
> > > > Embedded systems usualy place editable files on flash fs and those
> > > > fs get used out very much faster compared to real hdfs.
>
> I think that sed creates new file, fills it and and then renames
> new file into old file so fast that first metadata update doesn't
> even have time to hit the disk storage before it's all done.

I believe that may be the case with most fs, but not jffs2.  But don't
take my word for it.

> What exactly are you trying to optimize out here?

The shell?

> Why do you think your solution will be any different from what sed does
> now wrt storage wear?

I can't say I _have_ a solution.  I'm just thinking loud ;)

> > > I think you mean
> > >
> > > sed ... <"$file" >/ramfs/tmp-$$ && mv /ramfs/tmp-$$ "$file"
> >
> > If one would prefer using mktemp (mkstemp) instead of tmp-$$, it would
> > get more complicated and inefficient.
> >
> > > This is what shells are for - to avoid bloating programs with all
> > > imaginable small improvements.
> >
> > But of course.  That is one way to do it, but is it as efficient as
> > having sed do it in one go?  What about combining sed and mv code to
> > achieve the same?
>
> Well, what about -R option - do it recursively over a directory?
> and -find_name '*pattern*' to act only on a subset of files?

Sorry, I don't follow.  "-R option" to what?

> I don't like this kind of creeping featuritis.

I agree.  Still, there are a lot of embeded systems out there using
busybox and flash storage.

> Does stock sed do this?

No.  GNU sed is doing something similar to bb-sed (or the other way
around):

,----
| open("lines", O_RDONLY|O_LARGEFILE) = 3
| ioctl(3, SNDCTL_TMR_TIMEBASE or TCGETS, 0xbfd35068) = -1 ENOTTY (Inappropriate ioctl for device)
| fstat64(3, {st_mode=S_IFREG|0644, st_size=8, ...}) = 0
| gettimeofday({1188158729, 155778}, NULL) = 0
| getpid()                                = 19852
| open("./sedSTqTlj", O_RDWR|O_CREAT|O_EXCL|O_LARGEFILE, 0600) = 10
| fcntl64(10, F_GETFL)                    = 0x8002 (flags O_RDWR|O_LARGEFILE)
| fstat64(10, {st_mode=S_IFREG|0600, st_size=0, ...}) = 0
| mmap2(NULL, 131072, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7dfe000
| _llseek(10, 0, [0], SEEK_CUR)           = 0
| fchmod(10, 0100644)                     = 0
| fchown32(10, 1000, 1000)                = 0
| fstat64(3, {st_mode=S_IFREG|0644, st_size=8, ...}) = 0
| mmap2(NULL, 131072, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7dde000
| read(3, "fxx\nbar\n", 131072)           = 8
| write(10, "fxx\n", 4)                   = 4
| write(10, "bar\n", 4)                   = 4
| read(3, "", 131072)                     = 0
| close(3)                                = 0
| munmap(0xb7dde000, 131072)              = 0
| close(10)                               = 0
| munmap(0xb7dfe000, 131072)              = 0
| rename("./sedSTqTlj", "lines")          = 0
| close(1)                                = 0
| close(2)                                = 0
`----

But I don't think GNU sed was made with embedded systems and flash storage
in mind.

To mimic the `sed -i' (but with the temp file on a ramfs) I need a shell
script (or function) doing something like this:


---8<---
# $1 - expression
# $2 - file

if tmpf=$(mktemp /tmp/${2##*/}.XXXXXX); then
	sed -e "$1" < $2 >> $tmpf && mv -f $tmpf $2 || {
		rm -f $tmpf
		echo "E: you lose" >&2
		exit 1
	}
else
	echo "E: can't create temp file" >&2
	exit 1
fi
---8<---

That, I think, may take more cycles than:

sed -i -e "$1" $2


Cheers,

-- 
Cristian