[IPython-dev] saving base64 encoded content separately

[Chris Barker]

I have no idea if there's a new list -- but you're right, I haven't gotten
a note from this list in ages.

There's a whole pile of Project Jupyter community groups:

https://jupyter.org/community

maybe the discourse channel for the notebook is the way to go:

https://discourse.jupyter.org/c/notebook/31

but on to your problem:

There is a real challenge here -- the goal of .ipynb files is to store the
whole notebook in a single file, and that is JSON format, which requires
base64 or similar to embed binary data.

Yes, it would be fairly simple to have it reference an external file
instead, but then you'd have a bundle of files that needed to be kept
together. We could put them all in a single zip file, but that may well run
afoul of other overzealous security scanners :-(

I wonder if one could use BSON, which has a Binary data type, but is
otherwise a pretty one-for-one match with JSON.

Of course, maybe the security folks won't like that either -- frankly, you
either have arbitrary binary data or you don't :-)

-CHB



On Thu, Jun 3, 2021 at 11:03 AM Jonathan Hess <junkatar at vivistar.com> wrote:

> [The archives show this list has become very quiet.  If developers have
> moved to another forum, care to tell me where?]
>
> I'm looking for an option or extension that would, for .ipynb files,
> replace base64 encoded data with references to an external "native" file of
> the same data.
>
> For example, if an ipynb file has a matplotlib produced png image, I need
> that image stored in an external png file.
>
> Yes, integral base64 encoding is nice from network, efficiency, and file
> management aspects.
>
> Unfortunately, security conscious organization that have established
> review processes simply will not allow base64 content -- it is too easy to
> hide content or malware that way from their established (read unchangeable)
> processes.
>
> I've been reading up on:
>
>   https://ipython.readthedocs.io/en/stable/config/extensions/index.html
>   https://ipython.readthedocs.io/en/stable/config/shell_mimerenderer.html
>
> But, before I dive off into the trial and error time sync of just trying
> things:
>
> A) Has anyone already done an extension like this?  My websearches have
> failed so far but I would be surprized if no-one has asked about such a
> thing before.
>
> B) If A) is no, anyone have suggestions or guidance on making such an
> extension?
>
> Finally (and possibly not relevant to this forum), with such a feature or
> extension for ipython, I would expect end users to be able to control it
> thru a Jupyter Lab configuration setting.
>
> Thanks for reading this far,
> Jonathan Hess
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at python.org
> https://mail.python.org/mailman/listinfo/ipython-dev
>


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.python.org/pipermail/ipython-dev/attachments/20210603/2a2d2e9f/attachment.html>