Long time no post

Switching to a new computer and a new eMail has been nothing short of a time consuming disaster, not to say expensive.  I should start posting again this week.  See the previous post for why you must continually update your data to keep it accessible.

I do recommend an excellent review article on phase transitions in the cell [ Neuron vol. 109 pp.. 2663 – 2681 ’21 ] which tries to make sense of the chemistry behind it, particularly focusing on RNA.  Unfortunately it is likely behind a paywall.  I have written on the subject a while back, so here’s a link — https://luysii.wordpress.com/2020/12/20/neuroscience-can-no-longer-ignore-phase-separation/.  Now that people are looking for it, new examples are constantly being found.  It’s in chromatin. It’s at the synapse etc. etc.

Now a plea for help.  The hardest thing about shifting my database to Mathematica is finding a way to sort it.  This was no problem at all for Filemaker or Hypercard.  All you had to do was type sort and the programs did the rest.  Well no I have to do it.

Does anyone out there have any ideas what sort of internal data structures are available to keep 22K cards coherent and search them, while modifying them by continuing to read the literature.

No one will be able to access your data in 20 years

Hardware changes all the time requiring new software to use it even for the ‘same’ program.

 

Consider my history with HyperCard, a great Apple program I started using in 1987.  Apple didn’t support it after it moved away from Operating system 9 — new hardware managed memory differently.  So I was left with OS  X which still had a way to access OS 9 even though the processor was a 2.1 GigaHertz PowerPC.

Then newer hardware no longer accessed even the power PC, so as long my iMac G5 held out I was OK.  But hardware doesn’t last forever, so I tried to migrate my database to FileMaker Pro, another Apple product (although they tried to keep this quiet).

Filemaker documentation is simply horrible. Example: they don’t tell you what the reserved words are believe it or not.

So I’m currently trying to migrate my HyperCard database to Mathematica.  I had bought Mathematica 10 and the computer it runs on in 2015.  It was a macBook Pro with a 2.2 GigaHertz Intel Core i7.  But this year the battery started swelling (which Apple offered to replace for free), and worrying about exploding batteries in cars, I decided to move on.

So I bought the latest macBook Pro, which has 64 bit hardware instead of the 32 bit hardware of the old macBook Pro.  This means that Mathematica10 won’t work on the new computer so I must upgrade to Mathematica 12.

Well  what you get when you actually try to open a Mathematica notebook written in Mathematica 10 on the new machine is something like this.

(* Content-type: application/vnd.wolfram.mathematica *)

(*** Wolfram Notebook File ***)
(* http://www.wolfram.com/nb *)

(* CreatedBy=’Mathematica 10.0′ *)

(*CacheID: 234*)
(* Internal cache information:
NotebookFileLineBreakTest
NotebookFileLineBreakTest
NotebookDataPosition[ 158, 7]
NotebookDataLength[ 69720, 1756]
NotebookOptionsPosition[ 63597, 1546]
NotebookOutlinePosition[ 64410, 1576]
CellTagsIndexPosition[ 64240, 1569]
WindowFrame->Normal*)

(* Beginning of Notebook Content *)
Notebook[{
Cell[BoxData[
RowBox[{“(*”, ” “, “W645″, ” “, “*)”}]], “Input”,
CellChangeTimes->{{3.8400255875558558`*^9, 3.840025593274387*^9}}],

Cell[CellGroupData[{

Cell[BoxData[
RowBox[{“newList”, ” “, “=”, ” “,
RowBox[{“ReadList”, ” “, “[“, ” “,
RowBox[{
“\”\</Users/lewisrobinson/Desktop/ Cards 9461 to 9462\>\””, ” “, “,”,
” “, “string”}], ” “, “]”}]}]], “Input”,
CellChangeTimes->{{3.8400252429862013`*^9, 3.840025260918976*^9}, {
3.840025304055138*^9, 3.840025325036539*^9}}],

Hardly readable or usable is it?  Presumably the as yet unpurchased Mathematica 12 will be able to read this notebook and put it into recognizable form on the new machine.

Now let’s move 20 years into the future.  Further new hardware, further new software.  Will you be able to find a machine like my new computer

  Model Name: MacBook Pro

  Model Identifier: MacBookPro16,1

  Processor Name: 6-Core Intel Core i7

  Processor Speed: 2.6 GHz

  Number of Processors: 1

  Total Number of Cores: 6

  L2 Cache (per Core): 256 KB

  L3 Cache: 12 MB

  Hyper-Threading Technology: Enabled

  Memory: 16 GB

  System Firmware Version: 1554.100.64.0.0 (iBridge: 18.16.14556.0.0,0)

  Serial Number (system): C02G8AS4MD6M

  Hardware UUID: 28C836B3-C406-5215-AB85-A25653ADF226

  Provisioning UDID: 28C836B3-C406-5215-AB85-A25653ADF226

  Activation Lock Status: Disabled

 

In 2041 will you (or your grandson) be able to find a copy of Mathematica 12 to run it on, as the newer versions are unlikely to run on such an old computer (as just happened).

 

I seriously doubt it, cloud or no cloud.   So maintaining your data is a never ending process.

Logically correct operationally horrible

A med school classmate who graduated from the University of Chicago was fond of saying — that’s how it works in practice, but how does it work in theory?

Exactly the opposite happened when I had to do some programming. It shows the exact difference between computer science and mathematics.

Basically I had to read a large textEdit file (minimum 2 megaBytes, Maximum 8) into a Filemaker Table and do something similar 15 times. The files ranged in size from 20,000 to 70,000 lines (each delimited by a carriage return). They needed to be broken up into 1000 records.

Each record began with “Begin Card Xnnnn” and ended with “End Card Xnnn” so it was easy to see where each of the 1000 cards began and ended. So a program was written to
l. look for “Begin Card Xnnn”
2. count the number of lines until “End Card Xnnn” was found
3. Open a new record in Filemaker
4. Put the data from Card Xnnnn into a field of the record
5. Repeat 1000 times.

Before I started, I checked the program out with smaller sized Files with 1, 5, 10, 50, 100, 200, 500 Cards.

The first program used a variable called “lineCounter” which just pointed to the line being processed. As each line was read, the pointer was advanced.

It was clear that the runTime was seriously nonLinear 10 cards took more than twice the time that 5 cards did. Even worse the more cards in the file the worse things got, so that 1000 cards took over an hour.

Although the logic of using an advancing pointer to select and retrieve lines was impeccable, the implementation was not.

You’ve really not been given enough information to figure out what went wrong but give it a shot before reading further.

I was thinking of the LineCounter variable as a memory pointer (which it was), similar to memory pointers in C.

But it wasn’t — to get to line 25,342, the high level command in Filemaker — MiddleValues (Text; Starting_Line ; Number_of_Lines_to_get) had to start at the beginning of the file, examine each character for a character return, keep a running count of character returns, and stop after 25,342 lines had been counted.

So what happened to run time?

Assume the LinePointer had to read each line (not exactly true, but close enough).

Given n lines in the file — that’s the sum of 1 to n — which turns out to be n^2 + n. (Derivation at the end)

So when there were 2*n lines in the file, the runtime went up by over 4 times (exactly 2^2 * n^2 + 2n)

So run times scaled in a polynomial fashion k * n lines would scale as k^2 * n^2 + k * n

At least it wasn’t exponential time which would have scaled as 2^k

How to solve the problem ?

Think about it before reading further

The trick was to start at the first lines in the file, get one card and then throw those lines away, starting over at the top each time. The speed up was impressive.

It really shows the difference between math and computer science. Both use logic, but computer science uses more

Derivation of sum of 1 to n.

Consider a square n little squares on a side. The total number of squares is n^2. Throw away the diagonal giving n^2 – n. The number of squares left is twice the sum of 1 to n – 1. So divide n^2 – n by 2, and add back n giving you n^2 + n