jed-users mailing list

[2025 Date Index] [2025 Thread Index] [Other years]
[Thread Prev] [Thread Next]      [Date Prev] [Date Next]

Re: [jed-users] new_process() can't handle input file > 65536 bytes?


https://man7.org/linux/man-pages/man7/pipe.7.html

 Before Linux 2.6.11, the capacity of a pipe was the same as the
       system page size (e.g., 4096 bytes on i386).  Since Linux 2.6.11,
       the pipe capacity is 16 pages (i.e., 65,536 bytes in a system with
       a page size of 4096 bytes).  Since Linux 2.6.35, the default pipe
       capacity is 16 pages, but the capacity can be queried and set
       using thefcntl(2)  <https://man7.org/linux/man-pages/man2/fcntl.2.html>  *F_GETPIPE_SZ *and*F_SETPIPE_SZ *operations.  See
       fcntl(2)  <https://man7.org/linux/man-pages/man2/fcntl.2.html>  for more information.  Since Linux 4.5, the default pipe
       capacity is lower than 16 pages when the/pipe-user-pages-soft/
       limit is exceeded.

To make the routine work S-Lang needs access to get and set pipe size under Linux.

That's a long man page and I don't know how accurate it is since there are now an increasing number of user/account/process settings.


On 6/29/2025 10:16 PM, John E. Davis wrote:
Tom Culliton<tom.culliton@xxxxxxxxx>  wrote:
Yeah, but likely a system pipe buffer size.
Yes, that’s my understanding. The process will exit only after it has
successfully written all data to the pipe. In this scenario, it got
stuck because the pipe was full.

Thanks,
--John

On Sun, Jun 29, 2025, 16:10 Roland Hughes<roland@xxxxxxxxxxxxxxxxxxxx>
wrote:

Color me confused.

If he deletes one line from the scratch buffer of 65537 lines it works.

If the buffer is 65537 (or presumably greater) it doesn't.

How does it work with just one line less if cat is always waiting for
input?

It's okay to say I'm an imbecile here. I'm desperately trying to get all
of the page breaks and numbers into a new novel that will hopefully be
available for sale prior to the fourth, so might have missed something.
It sure sounded like he ran headlong into a hard coded 64K buffer size
somewhere.

On 6/29/2025 2:44 PM, John E. Davis wrote:
Morten Bo Johansen<mbj@xxxxxxxxx>  wrote:
If you have the time, then create a file, "t", of 65537 bytes

     $ for i in {1..65537}; do echo -n a >> t; done

In a *scratch* buffer, insert the following lines:

     require("process");
     variable obj = new_process(["cat"]; write=1, stdin="t");
     () = obj.wait();
The issue here is that the "cat" process is waiting for you to read
the data.  For example, add

     () = fgets (&str, obj.fp1);

before calling the wait method.  If you want the output of cat to go
to stdout, then omit the write=1 qualifier.  From the documentation:

     write=fds

      fds is a list of integer file descriptors that are open for write
      access in the subprocess, and may be read to by the calling
      process using the fdN or fpN fields of the structure.
[...]
    Mimic popen(pgm, "r"):

     obj = new_process (pgm; write=1);   % Read from obj.fp1

I hope this helps,
--John


and then:

     alt-x evalbuffer

In my case the process hangs/freezes.

If I remove one single 'a' from the file "t" to make the size 65536
bytes,
then it completes.

Is it a case of some integer data type somewhere in the slang source
files
that should be changed or is it just me who don't know what I'm doing?

Regards,
Morten
_______________________________________________
For list information, visit<http://jedsoft.org/jed/mailinglists.html>.

_______________________________________________
For list information, visit<http://jedsoft.org/jed/mailinglists.html>.

--
Roland Hughes, President
Logikal Solutions
(630)-205-1593  (cell)
https://theminimumyouneedtoknow.com
https://infiniteexposure.net
https://johnsmith-book.com

_______________________________________________
For list information, visit<http://jedsoft.org/jed/mailinglists.html>.


_______________________________________________
For list information, visit<http://jedsoft.org/jed/mailinglists.html>.

--
Roland Hughes, President
Logikal Solutions
(630)-205-1593  (cell)
https://theminimumyouneedtoknow.com
https://infiniteexposure.net
https://johnsmith-book.com

[2025 date index] [2025 thread index]
[Thread Prev] [Thread Next]      [Date Prev] [Date Next]