Team:BU Wellesley Software/Notebook/JanooNotebook

From 2011.igem.org

Revision as of 16:01, 28 July 2011 by Jtferns (Talk | contribs)

Contents

Life = 42

jtferns

My HTML scratchpad

Follow me on my brand-new Twitter: Sigma7707!!

--Jtferns 15:47, 7 June 2011 (CDT)


To-Do List

  • read through Doug's Algorithm paper [http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2860133/]
  • read through AssemblyManager code
  • read through ClothoHelp [http://wiki.bu.edu/ece-clotho/index.php/Main_Page]
    • edit main page to allow for click-able images
    • break main page
    • fix link-caption centering issue
  • read through kirigami/TB paper
  • read through Friedland abstract/paper [6/7/2011]
  • implement HTML in the wiki
  • create a sandbox/html-testing page
  • create a fully-functional interactive calendar
    • create a basic calendar
  • create a fully-functional navigation menu/bar
  • create an HTML notebook page
  • win the wiki War


PuppetShow Thought Project

  • read through Puppeteer/PuppetShow powerpoint [6/7/2011]
    • PuppetShow takes some version of the protocol graph produced via the Assembly Planner
      • The relationships between parts within the protocol graphs are assumed to be labeled with protocol names
      • The PuppetShow itself should link the protocol names to libraries and help generate the necessary Puppeteer code
  • read through Puppeteer/related abstract [6/7/2011] 
Using the Clotho platform [4], we develop two
applications for specifying and executing biological protocols.
The Assembly Planner [5] is the end-point of an endto-
end design workflow [3] that produces an assembly plan
for synthetic biological devices, with each assembly step
annotated with the name of a biological protocol. Each
such protocol itself may be fully specified using another
Clotho application called PuppetShow, which provides an
environment for writing, testing, debugging, and executing
biological protocols.
--A Software Stack for Specification and Robotic Execution of Protocols for Synthetic Biological Engineering
  • peruse GroovyScripter code/structure
    • read up on JEditorPane first!! [6/8/2011]
      • need to find out how to properly import Python shell to run Python code [6/9/2011]
      • able to successfully incorporate python interpreting using native Python 3.2 installation!
    • attempt to mimic GUI and incorporate Python-parsing? [6/8/2011], [6/9/2011]
      • able to successfully mimic the JEditorPane!!
  • figure out how to seamlessly integrate Python script-compiling within fluid GUI [6/9/2011]
  • design aforementioned fluid GUI
    • currently working on implementing fluid menu bar design for easy navigation and script creation/manipulation [6/10/2011]
  • need to meet with Viktor and Chenkai to discuss the integration of the Puppeteer stack components into PuppetShow
    • add button and feature allowing the user to import collections from the database (much like how Algorithm Manager imports collections) [6/16/2011]
    • add couple more buttons with specified features (TBD)
      • dropdown selections to represent different possible tree annotations (non-functioning for now; defaults to BioBrick)
      • tabbed panes (still working out several kinks)
      • generate-Puppeteer button (eventually will function off of dropdown selection)
  • comment most, if not all, of the PuppetShow code [6/17/2011]
  • Talk to Wellesley about GUI front-end
    • pending for Monday's meeting
  • Figure out Puppeteer workflow and functions of the proposed Run button
    • pending for Viktor's knowledge-transitioning plan(s)
  • Setup BioCoder
    • implemented BioCoder in VS2010
    • able to generate .htm file for Human-Readable protocol
    • unable to set up GhostScript/GSView to view the produced graphviz graph :(
  • Talk to Avi
    • currently expecting him to meet with our lab this Friday
  • Puppeteer code converted to some kind of human-readable format
    • should coincide with BioCoder implementation/integration/etc
  • Puppeteer code based off of kits?


Protocol Graphs Thought Project

  • Ideally, we should have a protocol graph that would be a generic form of managing an arbitrary combination of protocols.
  • The protocol graph should contain vertices, which are either operands or protocols/operations.
  • The graph should also connect operands to protocols.
  • Given access to a library of protocol specifications, one should be able to give an expression or an explicit protocol graph for parsing purposes.
  • One should be able to use PuppetShow (see powerpoint) to create an implicit protocol graph.
  • One should be able to reconfigure protocol graphs for optimization, etc.
  • Questions
    1. How do we want to construct this graph?
    2. Does the user input an expression?
    3. Should we have an interactive GUI to help user create custom protocol graphs?
  • Generate several protocol graphs with Avi's help along with some predefined BioCoder protocols
  • examine Puppeteer code to see if the core commands need modification to cater to Avi's needs
  • brainstorm Protocol Graph/PuppetShow ideas in correlation with BioCoder paper and relevant thoughts


Database Thought Project

  • use Hibernate Connection Library?
    • makes sense if I'm using Clotho, otherwise I just add the appropriate JAR file
  • use POJO objects to structure classes in compatible format
  • use MySQL database/table to play around with inserting database entries and querying records


The Day-to-Day

[6/8/2011]

PS Scripter GUI as of 6/8/2011

Finally got the plugin to properly load off of the Clotho dashboard. It runs as a JEditorPane with (I assume; has yet to be verified) Python-syntax-highlighting. Next up is finding a way to properly integrate a "run" method for parsing/executing Python scripts made within this GUI.

Got a bare-bones menu-bar working. It doesn't have any functioning implementations, yet. However I'm going to stop here until I can figure out how to parse Python and run it from the GUI window.




[6/9/2011]

I started looking into incorporating the Jython library/software package to assist with compiling Python scripts, etc. However, as per Swapnil's advice, instead I shifted to just trying to run the Python scripts directly off of a native Python installation. This makes sense since one would currently need to have Python installed to work with Puppeteer, so executing/compiling the Python scripts/code should work.

Right now I am trying to properly code the runtime-execution of python to parse some sample code within a test file that I placed within the App directory. My thoughts for future development involves saving the written code to a temp file and executing the code when the user chooses the run command.

Update: The App can now fully parse text written within the custom-JEditorPane and interpret it using the native Python 3.2 installation. I have currently abandoned any interest in incorporating Jython and will focus more on GUI design and simple file-manipulation.

[6/10/2011]

PS GUI as of 6/10/2011

The cloning process is complete. The PS Scripter can now fully open files, save them, save them as a new file, and run the script. I will continue to play around with more GUI options but I will be leaving this App/Module alone so I don't brick anything just yet. The next step involves creating some new App with a more streamlines GUI that has some kind of status window.

Update: GUI now has a neat status window. I am currently implementing this by appending Strings to the TextArea, but there might be a better setup for future development. Right now it only spits out the basic text that I relied on for debugging. Eventually, it would either incorporate the python output/error stream (if possible). It also has a non-functional progress bar that I threw in for fun; if I can't get it to work with compile/run-time progress I will replace it with a status text.

[6/13/2011]

PS GUI as of 6/13/2011

The GUI has been re-designed to contain a re-sizable bar to vary the height of the visible JEditorPane and JTextArea; these are the python-scripting area and status window area, respectively. The status window itself now features descriptive text (need to fix some alignment issues) and the direct output and error streams from the python-script-execution. The image to the right shows the new GUI as well as some example code and the resultant status window text.








[6/14/2011]

PS GUI as of 6/14/2011

Fully re-designed the menu-bar components such that they were easily-modified via the built-in Netbeans GUI development tools. This resulted in the newly-implemented hotkeys for the various menu-bar components. Additionally, a new feature was added: custom command input. This feature allows the user to input a custom command for the runtime process execution; this allows users to input custom flags/commands as needed to do more with their Python scripts. Unfortunately, the current setup relies on saving the code opened/typed into the PS window to a "testing.py" file, which is what the Python interpreter compiles by default.

Update: The runtime process now no longer relies solely on the "testing.py" for executing previously-saved code. Instead, it uses the selected file once the user opens/saves their work. This change is also accurately reflected within the custom command option, which either presents the default "testing.py" choice or the current file appended to the default run-command text.

An issue that I still cannot figure out concerns why Swapnil cannot see the Python syntax-highlighting on his machine. My machine as well as the CIDAR PC1 both detect and highlight the code accurately. Therefore, something within the JSyntaxPane is acting up on Swapnil's machine. Currently the module is set up to use the associated JSyntaxPane library, so I might try experimenting with just wrapping the most recent JAR instead.

[6/15/2011]

I was working on my HTML scratchpad until Viktor, Chenkai, and myself had our meeting regarding the design of PuppetShow.

For now, I am to focus on implementing a feature that allows the user to import collections from their database connection. I was told that AlgorithmManager already supports this, so that app will help with the proper integration techniques as well as help me familiarize myself with the Clotho API. I am currently re-designing the GUI to provide easy-integration for several features that we have discussed and will be providing more information later on.

[6/16/2011]

PS GUI as of 6/16/2011

PuppetShow now has a functioning (needs some error-debugging) import button that allows the user to import a collection from their current database connection. Some more functionality will be added in the future, but for now I will work on some code-beautification techniques.




[6/17/2011]

File:TabGif.gif
PS GUI as of 6/17/2011

Currently finishing up my effort to comment most of the PuppetShow code.

Update: Several new features were added. Chenkai's Puppeteer-generator code was implemented into PuppetShow, as well as a tabbed-view feature. A lot of it is still being tested so there are a couple visual glitches. For the most part collections can be imported and used to generate their own Puppeteer code.





[6/20/2011]

File:TabbedGif.gif
PS GUI as of 6/20/2011

Major tabbedpane-GUI design changes implemented. Opening, running, saving are now fully-functional and successfully tested. The only feature missing at this point is the ever-so-important close tab and close tab button; these will be implemented shortly.

The new GUI now features functional tabs. Each tab has its own Python-editor, but every tab currently shares the same status window. The tabs feature generic names sporting descriptive text that highlights how they were created, etc. They also will reflect the name of their associated file if it was a tab generated from the opening of a file or the saving of a tab.

All of the running-functions have been modified to support the new tabbedpane features, as well as the Puppeteer code-generation. When a user imports a collection, the status window reflects the text-based version of the import and stores the string in an internal field. When they choose to generate Puppeteer code, it opens a new tab with the parsed Puppeteer code already generated. Ideally this should allow a user to freely edit code and multiple files as well as freely manage Puppeteer code.

[6/21/2011]

File:CloseTab.gif
PS GUI as of 6/21/2011

Rather lengthy inter-team meeting with both Computational and Wetlab components discussed, so there wasn't too much time to work on features and improve on them. However, I was able to get the "close" feature working for the tabbedpane view. Therefore anyone can freely create and close out new tabs. The run/save features should still be functional for each respective tab; only the open and new features were modified/implemented correctly.

Comp-Wetlab Meeting details:

BU Collaborative Meeting #1

  1. Give a summary of what I am doing
    • I am currently working on the development of a tool called PuppetShow. Initially, it was designed to be a simple Python-editing tool that would assist those working with Puppeteer scripts (which are produced through Python scripts). Since then, it has evolved into a tool that combines several features into one application. For now, it allows the user to freely create/open/save Python scripts and execute them with the appropriate output and error streams. It also allows the user to import a database or local collection, with which they can generate Puppeteer code using Chenkai's Puppeteer-generating code.
  2. Explain how it fits into the overall iGEM project
    • PuppetShow fits into the overall iGEM project as a versatile tool that will eventually allow knowledgeable users to create custom protocols for wetlab purposes. This should facilitate the usage of the Tecan robot, which should help the wetlab team increase their efficiency and decrease their sources of error.
  3. Explain what my next steps (up to a month) are and my timeline for carrying these out
    • I will merge my to-do list with my expected steps in some nice format soon. Until then, my timeline involves:
      • Fix tabbed pane issues by this Friday (6/24)
      • Finalize/implement annotation selector by next Friday (7/1)
      • Use gathered details to optimize test mode and look into simulator requirements by Friday (7/8)
      • Hopefully optimize/clean up any additional features and have PuppetShow polished/presentable by Friday (7/15)
  4. Let everyone know anything I don't understand, anything stopping my progress, any equipment/resources I need, etc
    • CIDAR couch?
    • CIDAR microwave Thanks Evan!
    • CIDAR DSLR Camera >:D



[6/22/2011]

Had our first Computational Meeting with Doug; several details were discussed and plans were formed.

I was also introduced to [http://research.microsoft.com/en-us/um/india/projects/biocoder/ BioCoder] by Swapnil, who directed me to their [http://www.jbioleng.org/content/4/1/13 paper].

I have lightly read through the paper and was able to set up the BioCoder source code on my laptop. Eventually I would like to integrate it as a temporary feature for PuppetShow. I also need to figure out how to properly display the graph files (it is created using graphviz, but none of the Win7 64bit options currently work for me).

Meeting Notes:

BU iGEM Computational Meeting #1

PuppetShow's expected functions:

  1. Create a Protocol Graph
  2. Annotate a Protocol Graph
  3. Generate Puppeteer Code
  4. Perform certain operations on Protocol Graphs


Append to To-Do List:

  • Talk to Wellesley about GUI front-end
  • Figure out Puppeteer workflow and functions of the proposed Run button
  • Setup BioCoder
  • Talk to Avi


PuppetShow's possible implementations:

  1. "BioCoder" Human Readable Description
  2. Given Constraints
    • Yes?
    • No?
    • Consumables
    • Time
      • Start
        • ASAP/ALAP
      • Finish


PuppetShow GUI notes:

  1. Protocol Graph from a ClothoAlgorithm
    • Run-button function is still ambiguous
      • How to define?
      • Currently defined "hard-coded" on a Protocol Graph-basis
  2. User-generated Protocol Graph
    • Run-button should generate Puppeteer code based off of user-placed annotations, etc.
  3. Specific protocols defined/edited for use with previous note
    • Should have these protocols stored in some database


[6/23/2011]

BU/Wellesley Collaborative Meeting #1

Another meeting, this time with Wellesley (Orit, Michelle and Kathy) and BU teams. We got a nice update on what Wellesley has been working on and how we are going to move forward in the future. Our next meeting with Wellesley should help us figure out our overall goals and help us re-orient our tools to adhere to them.


BioCoder/PG Prep:

  • re-read BioCoder paper
  • meditate on PG ideas/purposes/usage
  • organize thoughts for easy-presentation during tomorrow's meeting


[6/24/2011]

Met with Avi and discussed initial Protocol Graph features and PuppetShow layout (both current and expected). A lot of thought was put into discussing his needs (mostly from a GUI-perspective), and there was some brief discussion on coding structure and methodology.

[6/27/2011]

Requested some clarification from Avi regarding the file-directory system and a basic representation of a Protocol Graph.

Wellesley Meeting #1

Met with Wellesley (they greeted us with pizza!!!) and discussed several ideas:

  • Microsoft Surface and it's applicable SynBio tools
  • Primer designer
  • Electronic Notebook

I hope to resume on improving PuppetShow until we can propose a solid coding structure for Protocol Graphs.

[6/28/2011]

BU Collaborative Meeting #2

  • discussed Comp Team's visit to Wellesley (in rather exquisite detail via Chenkai)
  • caught up to speed on Wetlab's issues

Swapnil mentioned the possibility of using a MySQL/Hibernate/POJO connection setup for creating/using a new database connection. I have currently set up MySQL, created a table, and I am currently working on creating the Java class to communicate my local server. Right now my only issue is whether I need to have this be a part of Clotho or as an independent class; I chose to go with the latter for now.

Update: I was able to get the Hibernate connection to work!! Successfully created a table, and then added two entries using a class. I admit, I used the help on an online [http://www.allapplabs.com/hibernate/introduction_to_hibernate.htm tutorial], but there was a lot of self-teaching involved. Now I just need to figure out if Swapnil wants this to be a part of Clotho or independent, or if it's just an intermediate issue to understanding connections, etc.

[6/29/2011]

BU iGEM Computational Meeting #2

Swapnil tasked me with a new configuration window/setup that he would like me to implement.

Stack Configurations:

  • uses the Puppeteer Stack image
  • has an associated field to run a command-line per layer within the stack
  • has a sequencing option that lets the user choose when to run a sequence
  • should be attached to a config file that can easily be modified, etc. (not implemented yet)
  • has a tabbedpane status/output window for debugging, etc.
    • need to figure out how to incorporate user input for Viktor's python script

[6/30/2011]

Stack Configuration is mostly complete. It has fully-functional checkboxes, along with a load/save configuration option. Most of these features will need to be expanded on, but for now they will suffice.

[7/1/2011]

7/1/2011
7/1/2011

Just set up a query function to properly retrieve entries from my localhost MySQL database. Using this setup, it shouldn't take too long to set up a nice interface within ResourceManager to properly communicate with the eventual MySQL Puppeteer database.

I will start a wiki-documentation on how to properly set up HCL/POJO to talk to a MySQL database within Netbeans soon enough.

Update: Added close-tab feature to Stack Configuration output/error window; screenshot will be uploaded soon.










[7/4/2011]

Compiled (what I assume to be the correct) sequence video of video clips from the first successful restriction digest test via the Tecan robot and Puppeteer. Hopefully I will have enough spare time to trim this video (~27mins long) and produce a nice video.

[7/5/2011]

Fixed up minor bugs with PuppetShow custom-command execution and NullPointerExceptions.

BU Collaborative Meeting #3

  • learned that the Wetlab members are currently having issues with Ligations
    • they are currently employing several different methods to get appreciable results (16-degree incubation overnight, etc)
  • DNA concentrations need to be higher for specific tests, etc.
  • Wetlab wants to actively adopt higher forms of organization
    • ideally would like to integrate Clotho with their work (Betterboard, etc.)
  • Margaux appointed as Social Chair for fun movie/potluck weeknight!


Working on researching the Java console and figuring out some kind of solution for user input.

Verified that PuppetShow is functioning on Chenkai's MBP.

I was also keeping up with Avi via email conversations; it is very likely that he will stop by tomorrow to discuss ideas behind protocol verification and anything else necessary (such as where to hook up the Tecan simulator, etc.).

[7/6/2011]

BU iGEM Computational Meeting #3

  • need to schedule a "code-review" day for evaluating and maintaining good coding structure
  • have to figure out proper link between Puppeteer core commands and soon-to-be protocols
  • need to set up PuppetShow on Tecan computer
  • currently working on creating the underlying GUI infrastructure for protocol graph editor
  • figure out how to properly keep track of physical sample history
  • eventually work on verification between protocols and samples

[7/7/2011]

BU/Wellesley Collaborative Meeting #2

Discussion:

  • Social Event
  • Outreach
  • Next 5 ways for Collaboration
    • Trumpet Front-End Design
    • Notebook "demo"
    • Primer Design
    • Protocol Graph/Puppeteer link to notebook
    • Clotho interface

[7/8/2011]

Just set up an independent PuppetShowDB Java Application to properly set up and configure HCL to connect to a local MySQL database.

Update: PuppetShowDB can now properly query from the assigned table in a nice QueryView.

[7/11/2011]

Need to work on adding the "delete" and "update" features of MySQL into PuppetShowDB. Also having a lengthy discussion related to Protocol Graph class-structure and sample history with Swapnil and Chenkai.

Testing a page for Swapnil: My Database Information Page

[7/12/2011]

BU Collaborative Meeting #4

  • ligations should be back on track
    • the issue was not with ligations themselves but most likely related to cell-competency
  • I need to be able to explain graphs/Protocol Graphs to general audience
    • tasked to create a mockup (PowerPoint?) detailing expected GUI and how graph structure would work


Current agenda:

  • trying to set up a nice database-integration for PuppetShow
    • database will house Protocols and Samples
    • succeeded in inserting and querying from localhost database
    • aiming to figure out how to properly incorporate deletion and updating
  • had lengthy discussions with Swapnil and Chenkai regarding Protocol Graphs
    • discussed class infrastructure complexities
    • pondered methods of saving/recording sample history
    • configuring tables for ideal data structure setup

Update: During our discussion, we were analyzing the table-structure that Swapnil and Viktor were discussing a while back. Swapnil mentioned that he doesn't like the visual redundancy in the proposed ProtocolNodeTable, which inspired me to try to create a "symmetric" ProtocolNodeTable and SampleNodeTable. Both table-structures should work, but Swapnil tasked us with determining the total number of queries required for the specific tree I proposed as well as a generic tree with n-nodes.

Having analyzed both table-structures, I counted up to 8 queries for both structures. However, if you needed to be explicit and grab SampleNodeTable data between queries (such as the SampleNodeTable.SampleName for defining the Sample Node when reconstructing the graph), the total number of queries would increase significantly.

Therefore, for the very general case, the proposed query counts are:

In-and-Out table-structure: Sample with N-parent-nodes needs N queries (vague) or up to N+S queries, where S represents the number of parent-Sample-Nodes (explicit to the same level as "Symmetric").

"Symmetric" table-structure: Sample with N-parent-nodes needs N-queries (explicit, since every query pulls up relevant information about both Samples and Protocols :) ).

[7/13/2011]

BU iGEM Computational Meeting #4

  • continue working on tweaking independent PuppetShowDB file-saving issues
  • get output to appear on proper tabbed-output window
  • configure mysql properly!! (been pending for a while)
  • attempt to show a proof-of-concept database/svn run

[7/14/2011]

BU/Wellesley Collaborative Meeting #3

  • Wellesley projects
    • e-Notebook?
    • G-nome surfer work
    • Trumpet front-end for the MS
  • BU Wetlab awaiting new cells

PuppetShowDB's Run button needs to allow the user to simply open up a Puppeteer script and be able to execute it via Puppeteer/ResourceManager/Hal...

[7/15/2011]

Modified Run method to use pointed paths to start up Hal and ResourceManager. However, I forgot that we agreed on just having those two parts start up automatically with PuppetShowDB.

Implemented a killProcess method that is called for Hal and ResourceManager; they're called when the user exits PuppetShowDB.

Fixed a wrapped-JAR issue that required the user to keep pointing to an arbitrary mysql-connector JAR file (most likely implemented when I auto-generated the HCL config and mapping files within Netbeans).

[7/16/2011]

Attempted to resolve issue with calling ResourceManager within PuppetShowDB; it was giving a forced-disconnect error that was not entirely clear.

Also attempted to figure out new scriptWindow and scripTab classes to resolve file-saving issues with the tabbedpane-view.

[7/18/2011]

PuppetShowDB GUI is now cleaned up and effectively looks like a generic-script editor for Puppeteer (python) scripts.

TabbedPane was removed, as well as several related features (new, close, running python, test-mode).

PuppetShowDB also starts up ResourceManager and Hal on startup via the stackconfig.properties file locations.

[7/19/2011]

PuppetShowDB had some minor bugs with the modification of saving/opening files that were just fixed. Today starts the process of integrating ResourceManager seamlessly with PuppetShowDB. For now we are using a wrapped-JAR file approach to see if we can get ResourceManager up and running properly. We are currently experiencing difficulties with class/package specification issues.

[7/20/2011]

The past couple of days were spent trying to integrate ResourceManager, and every method (wrapping JARs, importing packages, creating packages, combining source files, etc.) has not worked. The main issue was with a NoClassDefFound error when running a test-file (as specified by Chenkai); Chenkai is working on trying to figure out why his code is having this issue, etc.

[7/21/2011]

BU/Wellesley Collaborative Meeting #4

  • ligations are working
  • wetlab folk are using Clotho (with some minor database issues)
  • wellesley visit
    • primer designer (soooo pretty?)
    • trumpet (also pretty)


Update: Was able to almost fully-integrate ResourceManager into Puppeteer after Chenkai fixed the bug. Turns out that ResourceManager has an issue with the ligate.py protocol, so running without ligation actually lets it work perfectly (why are ligations so troublesome?!).

[7/22/2011]

Finally able to properly integrate ResourceManager with PuppetShowDB.

[7/25/2011]

Tweaked PuppetShowDB GUI for ease-of-access/use and allowed for proof-of-concept output window generation.

[7/26/2011]

Turns out we will attempt to generate a nice and clean HTML report for proper viewing of the executed protocol (resource info, instructions, event log, etc).

Most of the HTML code is being generated without any formatting, but some simple css stylesheet(s) will take care of that.

[7/27/2011]

Visited Ron Weiss's lab at MIT to work with Jonathan to deploy Puppeteer software suite (Puppeteer scripts and PuppetShowDB GUI); deployment is still pending due to technical difficulties.

[7/28/2011]

BU/Wellesley Collaborative Meeting #4

  • Wellesley working on G-nome Surfer Pro(karyotic?)
  • are done by 8/5
  • will work with Craig on MS Trumpet

The Backburner

OpenPCR

  • Create a modularized PCR for easy integration with the Tecan robot
  • costs ~$500, which is almost 10x-15x less than commercial cost
  • need to look into open-source code and see their implementation methods


The Guestbook

Feel free to leave (appropriate) comments, suggestions, questions, notes, etc., here:

Retrieved from "http://2011.igem.org/Team:BU_Wellesley_Software/Notebook/JanooNotebook"