Teaching Series #4 - Scientific Methodology

At our core, we are all scientists. In one way or another, we keep asking simple questions, like how aeroplanes work or what fruit is the sweetest; we all have an inquiry. First step, congrats!

However, to truly let the scientist in you resurface, that inquiry needs to be systematic! Systematic inquiry means that you gather information from multiple sources (that is called information gathering). And you don’t stop there! You design a method (e.g., a survey or an experiment), collect and analyse the results statistically, and then draw conclusions about observations, that's research, that’s science!

The main purpose of science can be classified as follows:

• Discovering new knowledge: Think of scientists discovering new planets or developing new medicines, algorithms… anything that no one has heard or seen before 🙂 

• Verifying existing knowledge: More often than not, we need to double-check if what we think we know is still true, or if it holds up under different conditions. It's like re-running an old experiment to confirm the results. Everything is evolving! Sometimes more rapidly than you think 🙂 

• Solving problems: Most often, research and researchers can help us find solutions to real-world issues, from desalination of water to designing more efficient technologies.

General process of research

Given many different factors, like the domain of research, the specificity of a project, research methodology - or call it a protocol - things may look very different. But they usually follow a similar path:

• Identifying a Problem or Question: This is where it all begins! You notice something you don't understand, or there is a gap in knowledge, or a real-world issue that needs a solution. For example, you might wonder, "Why do some plants grow faster than others in the same conditions?"

• Collecting Data: Once you have your question, you need to gather information. This could involve experiments, surveys, observations, reading existing books or articles, or even talking to people. If you are studying plant growth, you might measure plant heights daily or test different types of soil.

• Analysing Data: After you have collected your data, you need to understand what information or knowledge is hidden in that data. This involves looking for patterns, trends, or relationships. If your plant growth data shows that plants in one type of soil consistently grow taller, that is a pattern.

• Drawing Conclusions: At the end, you interpret what your analysis tells you and answer your initial research question. What did your data reveal? Did it support your initial ideas? You might conclude that a certain soil composition significantly impacts plant growth.

Beyond the satisfaction of bringing benefit to the entire society, engaging in research also offers significant benefits for researchers:

• Critical Thinking and Problem-Solving: Research forces you to think deeply, analyse information, identify problems, and come up with creative solutions. These are invaluable skills in any field, especially in computer science, where complex problems are the norm.

• Intellectual Curiosity: Research is all about exploring the unknown. It feeds your natural curiosity and helps you develop a lifelong love for learning and discovery!

Where do you start?

Well, ideas do not fall from the sky! You wouldn't try to invent a new type of computer chip without first understanding all the existing chip designs, right?

The literature review is the place where you start systematically finding, evaluating, and synthesising all the existing scholarly work related to your research question.

• Understanding Existing Knowledge: The most fundamental reason is to see whats already known about your topic. You need to understand the theories, findings, and debates that have already taken place. This prevents you from "reinventing the wheel."

• Identifying Gaps in Knowledge: This is perhaps the most exciting part! As you read, you will start to notice areas where current research is lacking, where there are conflicting findings, or where new questions have emerged. These "gaps" are often where your own research can make a significant contribution.

• Refining Your Research Question: Sometimes, after reviewing the literature, you might realise your initial question is too broad, too narrow, or has already been answered. The literature review helps you fine-tune your question to ensure it's original and impactful.

• Learning Methodology: You can also learn from how other researchers have tackled similar problems. What methods did they use? What were their challenges? This can inform your own research design.

You approach the literature review by searching databases. Use academic databases (like IEEE Xplore, ACM Digital Library, Google Scholar for computer science, or broader ones like Web of Science, Scopus) with keywords related to your research question. You need to Read Carefully and Think Critically. You aim to Identify Ideas and discover the Literature from a Different Perspective.

What is Methodology?

Your methodology is your justification for the specific surveys, experiments, interviews, etc, you choose to use.

If your question is "How many users prefer feature A over feature B?", your design will likely involve a quantitative experiment or a large-scale survey. Quantitative methods are all about collecting and analysing numerical data to identify patterns, test hypotheses, and make predictions

If your question is "What are the common frustrations users experience with feature X?", your design will likely involve qualitative interviews or observations. These are the methods that provide descriptive insights into phenomena.

Choosing the "How” you are going to conduct your research depends on your research questions!

For example, for a quantitative study on algorithm performance, your design might be an experimental design. Your methodology would justify why an experiment is the best way to measure objective performance metrics (like execution time, memory usage) and compare them. You would then detail the specific methods: writing code for both algorithms, defining test datasets, setting up a controlled testing environment, and using performance profiling tools.

For a qualitative study on developer collaboration in open-source projects, your design might be a case study or an ethnographic study. Your methodology would explain why deep dives into specific projects and understanding social dynamics through observation and interviews are necessary. The methods would include: conducting semi-structured interviews with project maintainers, observing communication on forums/GitHub, and analyzing commit messages.

Essentially, the research design is your comprehensive plan, and the methodology explains and justifies the specific methods you will use to execute that plan. It's all about ensuring your research is rigorous and can genuinely answer your question.

The final step is often the hardest 🙂 

What is the point of doing all the aforementioned hard work (identifying a problem, planning, collecting data, and analysing it) if no one else knows what you have found? Communicating your research is how you contribute to the collective body of knowledge and allow others to benefit from your discoveries!

Research Findings are communicated through:

• Writing Research Reports/Papers: This is the most common and formal way to share research. A research paper typically follows a standard structure:

  • Abstract: A summary of the entire paper.

  • Introduction: Introduces the problem and states the research question. In its core, it captures the attention of a reader! 🙂 

  • Literature Review: Summarises existing knowledge and highlights the gap your research fills.

  • Methodology: Explains how you conducted your research (design, participants, data collection, analysis). This section needs to be detailed enough for others to potentially replicate your study!

  • Results: Presents the findings of your data analysis, often using tables, graphs, and statistics.

  • Discussion: Interprets the results, relates them to your research question and the literature, discusses limitations, and suggests future research.

  • Conclusion: Summarises the main takeaways! What is something that you want a reader to remember? Think about human constraint, we can rarely remember everything!! 🙂 

  • References: Lists all sources cited. It is of utmost importance to give praise to those people who inspired you!

Researchers often present their work at conferences, workshops, or seminars. These presentations are typically shorter and more visual, aiming to convey the key findings and generate discussion.

For academic degrees (Masters, PhD), research is typically compiled into a comprehensive thesis or dissertation, which is a much longer and more detailed document than a journal or conference paper.

Good communication allows other researchers to understand your methods and results, enabling them to build upon your findings, replicate your work, or explore new directions of research. In the same way that you build on top of someone elses work. In todays world, it is impossible that you started from scratch. If nothing, you are using someone elses pen or keyboard on which you are typing! Be grateful and appreciative of the sources you are using! 🙂 

This is how scientific progress truly accumulates. And with these simple couple of words, we have covered the entire journey of how research works and changes our lives for good!

Sincerely, MO