The rwx cycle [note-0048]

Autoresearch weaponization to ingest hundreds of papers in hours, run in-silico experiments, and write up 5000 words in an afternoon when you're too broke to afford journal subscriptions. Tested and true.

1R: Literature review

Jiabin Huang research advice offers a variety of tips for researchers.

First, perform a literature review.

Boiling the Ocean method: There are various "awesome-xyz" premade lists on GitHub that you can use to find papers. Consult past conferences to see where the field is heading. Cluster these papers into categories to understand "why x hasn't been solved yet".

Availability heuristic: Since many papers exist on X, you can just ask Grok to search for threads. Alternatively, look online at your favourite theorists' websites, as well as non-traditional researcher websites, and see if there are any recent writings that haven't been implemented yet. For example, I have implemented [undisclosed] suggestions from gwern.net for a research project, which isn't widely seen in academic literature.

Then, feed the context into your favourite research agent, in this case, I use Feynman developed by my mutual Advait and have suggested the following prompt contributions.

Feynman prompts sorted into different levels of granularity:

  1. [insert link] look around! As a leading expert in neuroscience, explain this lab's research canon and how it situates in the contemporary work ranked by originality, then point to individual significant papers of interest.

  2. You are a research methodology expert. For each paper, identify the major vs minor contributions buried in this data, rank them by originality, and show with precision and accuracy where each one challenges or extends existing literature. Use bold text very sparingly for notetaking in Logseq. Provide a glossary for a technical 20 year old. For each formula, provide intuition for reconstructing it step by step, with patterns understandable to a technical 20 year old. Do not use bold text.

  3. Still acting as the research methodology expert, explain from the primitives level how they designed the pipeline, the quality and type of evidence and reasoning used, and consistency.

  4. Michelin-tabling question (if you're meeting with the authors) As a leading neuroscientist, you find yourself at a fancy lunch with the authors. Read the discussion or looking forward section while thinking granularly. What are the 3 quirky follow-up questions that you would discuss with the team? What are the 3 key implications of this argument that you would encourage the authors to think about?

1.1Researcher archetypes

Identify your default research style and use it to your advantage!

  • The deep-dive specialist (months/years on one breakthrough)
  • The strategic collaborator (connecting dots across projects)
  • The methodology expert (enabling others' discoveries)
  • The translator (making research accessible and actionable)

2X: Scorched earth hypothesis testing
  • If you have around a week for a project, you should start running experiments ideally on day 1 or 2 to leave time for adjustments. Notice and resist the urge to execute all experiments on day 4.
  • 50% of the time should be spent on running experiments, logistics, or building.
  • Code should be constantly running.

Representations of the World in the Human Brain by Jack Gallant, an experimentalist.

According to your "why x hasn't been solved yet" buckets, continue narrowing each bucket down to testable and plausible hypothesis lineages. Come up with a list of hypotheses (anywhere from 1 to even 40) ranked by upside risk and time taken to test.

As soon as we have the hypothesis, we can scope the experiment. Tetraslam's playground has a great style guide for agents. You should also write a markdown style guide.

Sketch a diagram of how information flows through the key components, starting out simple. Pick your tools accordingly for each stage. Ask agents to contrast tools by community size, features, cognitive complexity, and pick the most relevant one for your use case.

It is a delicate balance to understand the experiment you are running vs optimizing for speed. You will feel cognitive strain. Adjust mental impedance accordingly. Remember: Too much speed can lead to accumulated technical debt exponentially further down the line. (See A Philosophy of Software Design by John Ousterhout.)

Put all the observations and graphs into a page and share it with your collaborators. If an experiment didn't work, why? Brainstorm the most probable explanations. Review related literature again, adjust the direction, and proceed to test.

Models need to be evaluated. Consult vals.ai

2.1Collaborators

Take advantage of mornings, lunch, evenings, walks to update results and brainstorm. Do not block others from doing their own work!

Meetings should not be the first time results are presented, since almost no one will digest them live and provide insightful suggestions.

3W: drafting papers

3.1Monofocus

If you are researching at a small startup, it is extremely difficult to simultaneously run side projects and not recommended. The founding team has extremely high stakes over a period of years and sometimes devotes 120 hours per week to ensure success. Founders may have given up $250k/yr jobs at companies to build their own startup, or even social time. You will notice that the singular research project undergoes mitosis and contain a multitude of sub-projects itself as time stretches.