DevOps

  • git (2)
  • response (2)
  • error (2)
  • gson (1)
  • http (1)
  • code (1)
  • status (1)
  • test (1)
  • junit (1)
  • rest (1)
  • A space dedicated to all those things you need to know beyond writing code. This covers Linux, Windows, Git and SVN among other hopefully useful nuggets.

    ERROR 1698 (28000): Access denied for user 'root'@'localhost

    If you have gotten this far you might be pulling your hair out. I know I was. This answer is plain and simple. The Ubuntu installation of MySQL uses the unix auth_socket plugin and so requires sudo if you are not the system root user. sudo mysql -u root -p

    RestAssured use Gson by Default

    Every now and then you come across a project where it is already using an JSON object mapper. In this can when you want to use RestAssured you should probably take advantage of the work or the libraries already in use in the project. Here I outline very quickly how to select your ObjectMapperType.

    Send a HTTP response code

    Postgres generate UUID for insert statement

    Out of the box postgres does provide a means of supporting UUID as a type but it is not enabled by default. I outline the steps required to enable the extension, create and alter tables to use the extension and finally how to use this as part of an insert.

    Reset Local Branch To Match Remote Branch

    Occasionally there is a need to blow away all the local changes, possibly they are no longer required. What ever the reason it is good to know how to get your branch back to matching the remote version. This can be summarised as reset and clean.

    submodule deinit

    To remove a git submodule it take a few commands. Replace [submodule path] in the commands with the appropriate project path.

    # The initial command git submodule deinit cleans the project entry from the .gitmodules file.
    git…

    Postgres Install as Service, Create a Database, User and Assign Privileges

    To install PostgreSQL on Ubuntu you run the apt install command. This will install the latest version of PostgreSQL if you want to run this as part of a non-interactive script add -y to accept the application and it's dependencies.

    # install postgres server and related dependencies
    $> apt install -y postgresql
    

    Using systemctl,…

    MySQL Create a Database, User and Assign Privileges

    To do this you will need to have Installed and configured MySQL.

    If you want to create a database and assign a user permission to access it you first need to login with a user who has permission to do this action. This is often the database root user. Run the following command. I am using the user root…

    Check if a port on a remote system is reachable

    If you wish to check a single or multiple tcp or udp ports is open from a linux system. You can leverage the nc (netcat) utility.

    MySQL Uninstall

    First I will say that if you are here to uninstall because you have lost access seeing error messages like ERROR 1698 (28000) the start with Access denied for user. If you would like to set a root password or clean up your install try this MySQL install.

    MySQL Install & Initial Configuration

    I do this every few months and I am always scratching around for my notes and the correct incantations. I have outlined the installation steps, configuration of root database password, schema creation and granting of user permissions. I hope you find this useful. MySQL 5.7

    SVN Essentials

    What is Version Control? Version control, software versioning or revision control systems are used to maintain current and historical versions of files such as source code, web pages, and documentation. Source code management using a version control system guarantees the follow. Subversion keeps the files in collections called Repositories. These repositories have a standard form which is outlined below. What does a Repository look like? As part of a standard repository you get three folders; trunk, branches and tags.

    Run Two Virtual Hosts The Same Port

    Using Apache is it possible to run two or more virtual host on the same port. I often use this as a means of web proxy for internal servers. Expose only port 80 on a proxy server and pass the requests to the correct internal host. Whatever your reason is, it is straight forward using the following configuration.

    Install the latest version of Maven on Linux

    This is a quick update to outline how to install the latest version of Maven on Linux. In the example below I have outlined the script required to install maven on Ubuntu and RedHat without a package manager. If you are running an older version of Ubuntu (14.04) you will not be able to access the latest version from the package manager.

    This is broken down into installing and linking the new version. I have linked out to my source repo for some bash scripts which you can run if you are keen.

    SSH Key Create and Access

    Security is paramount and one of the most secure means of accessing a system is using SSH Public/Private keys. I will outline here how to create a key, place it one the server and how to access the server using the private key. I generally create the keys on the destination and then copy the private key to the clients. It is essential to set a password for the keys, otherwise you could leave yourself (or the server owner) open to malicious access. So with that said we begin.

    Command Line User Management

    Every time I go to configure a new system for a client I stumble over the user management.  It is one of those areas that you work on and then leave for another 6 months.  I am sick of searching for each of the actions to manage users; so I have brought them together here.  This is aimed primarily at Ubuntu users but can be applied with minor tweaks to other systems.  In the example below I will be creating a user named tomred, and so any reference to tomred going forward will be to a user by that name and the hostname is ubuntu.example. (just for clarity)

    Network Error Connection Timed Out

    So you have seen the Network Error: Connection Timed Out when attempting connecting to your EC2 instance. Like me I am sure you will have checked everything 40 times and now be banging your head against the wall. Well fear not there is two last things to try.

    Remove Local Untracked Files and Directories From My Current Branch

    Occasionally you will want to remove changes from your local working copy. This can be because there were files or directories that were created that are no longer needed or branches were merged but contained files the were untracked.

    Create New Branch From Old Commit

    If you want to create a branch from an old commit you have a number of available options. In each case while the process is slightly different the result will be a branch from your desired point in time.

    Remove Server Header

    The default state of an NGINX server is to have the Server set. This returns something like Server: nginx/1.12.1. Here is a list of all the published exploits for nginx to give you an idea why you might not want Server header and the version published:

    Nginx Essentials

    No matter the problem

    Nginx Create htpasswd file

    Running the following command will prompt you for the password and then prompt you to confirm the password. The end result will be a username and a hashed password separated by a colon (:) in your .htaccess file.

    How to calculate concurrent users for a load test

    When you want to perform a load test you need to know and understand your baseline metrics. To help set that baseline calculation you need to know a couple of numbers.

    concurrent users (cu)
    This is how many people are using the system at the exact same time. This is usually much less than the total number of users that will be using the system.
    average user/session…

    Git Essentials

    This is a list of this command that you need to know to be effective in git. Like I always say this is not comprehensive but it should be enough to get you through most problems.

    Restarting nginx fail

    When calling sudo service nginx restart results in Restarting nginx [fail] it is time to test your config.

    The Pipeline - Continuous Integration and Deployment

    I have been working over the last decade to simplify software development and improve the quality of products delivered in each of the companies I have worked. I have continued to redefine what I refer to as "the pipeline" from definition of work through to delivery and installation in production.

    I won't get into whether you should be using agile or not or even which flavour. This is intended to cover the technical elements and not your SDLC or even the business culture.

    Different Levels in Automated Testing

    Breaking from the old war of unit verses integration tests and the slightly newer battles of TDD verse ATDD, I prefer to consider the tests by their size and their purpose. Automated testing can be modelled as three pyramids.

    Git Edit Commit Message

    If you want to edit an incorrect commit message that has not yet been pushed to the remote repository you can --amend the commit. This permits to to pass a -m message or edit it in your chosen editor.

    Git Revert a Commit

    If you would like to remove a commit that exists in the remote repository this can be accomplished by reverting. Reverting creates a new commit which is the reverse of the commit you are removing. This does not return the repository to the state before the commit but simply create a new state of the repository with this new commit.

    Git Move Last Commit to Another branch

    To move a local commit to a different branch you start with and undo a then you can do the following:

    Git Ignore Files

    To ignore files in git you can update the .gitignore file or update the index with assume-unchanged.

    Git Undo the Last Commit

    To undo a commit without loosing the changes maybe you missed a file or added something incorrect, then you would do the following:

    Windows Restore Show Desktop Icon

    To restore the show desktop icon to the Quick Launch menu of the Task bar Click Start button. Select Run and type the following command:

    Windows Junctions Symbolic Links

    Anyone how has worked for any amount of time on a Linux or Unix system will be aware (and most likely singing the praises) of symbolic links. The question often arises how to accomplish this in a windows environment.

    Windows How to Turn on Hibernation in Windows Vista

    I have come across this on a number of occasions, I do a disk clean up tell it to remove the 2Gb+ of Hibernation files and I can no longer enable hibernation. Why? and how do I enable it again?

    Windows Dos Command line Tab Completion

    Anyone who is familiar with linux/unix will be aware of the tab completion features when using the command line. Windows also provides such a feature but it is disabled by default. To make this change you will be making a registry change. You should save a copy of the registry before making any changes. You may need to log in as an Administrator to make these changes.

    Windows Deleting a Service

    It is sometimes necessary to manually remove/delete a windows service, this may be because during uninstall the application did not remove it or because the source was malicious. In any case this should be the last resort as it involves editing the registry.

    SVN Shallow Checkouts

    I have come across this issue a number of times where by I only require a few of the sub folders of a repository project in Subversion (svn). I always have to go searching to remember this so I have created this tutorial with the intention of sharing this as well as having some reference in the future.

    SVN .svn Missing Error

    You have come across this error svn: Directory .svn containing working copy admin area is missing the reason is simple. You have deleted the .svn directory (probably not on purpose).

    SVN error resolving case

    As with most errors experienced when using a application this error is less then self-explanatory. That said it is a simple type error. svn: error resolving case of is caused by the incorrect use of slashes. I have come across this a number of times when working on windows.

    SVN Set HTTP Proxy Info

    If you work behind a web proxy and you wish to do a checkout using the command line with SVN you have to set a couple of values to make subversion aware of the proxy.

    SVN Item or Property Statuses

    When working with subversion (svn) you often bump into Item or Property Statuses. Most of them a obvious but every now and then 'G' or '~' pops up and I have to look it up. I have included this page here to list this values so that they maybe of use to anyone else who may be looking for them.

    Install a jar as a service

    It doesn't matter if you are using the latest springboot or have an old runnable jar around. It is often useful to have an application installed as a service. This allows your application admins to use simple service app start commands to control your application.

    sudo: sorry, you must have a tty to run sudo

    If you are sending a sudo command across SSH you will often see this message. There are a number of ways to resolve this and I have outlined them below. The reason for this message is a property of the sudoers file called Defaults requiretty which is enabled by default on many linux distros.

    Vi Vim

    When working in a linux or unix environment it is practically impossible not to eventually use vi or vim. This is one of the first command line applications I used back in my Perl scripting days. Even to this day I still enjoy the simple approach but every now and then I forget one or two of the commands or shortcuts and so I have created this page.

    Tar Untar Gzip Ungzip

    Finally I've had enough; I do this at least 10 times a month and I can never remember what to do there is just two many other things to remember so I have created this as a reference so I don't have to resort to the ever knowledgeable Google for the answers.

    How to increase Swap

    This is a quick article on how to allocate more swap on a Linux system.  I have done this on multiple versions of Ubuntu but that is not to say it won't work elsewhere.  Before I start, you might ask what is swap?  Well it is a temporary space where the OS can drop information that is usually held in RAM when it is not needed so freeing up for other applications.  On a Linux system you can see how much you have using the free command.  This shows memory (RAM), buffers/cache and swap.  This is also the order in which the resources are used.

    Find Large Files in Linux

    There are a hundred different ways to finding large files in linux. My most regular one is because one of my VMs has run out of space. This is often caused by poorly managed log files or just general usage logs etc. What the real pain is trying to find the biggest offenders. I am not going to dictate which is the correct way but outline a couple of useful ways and hopefully if you have some of your own you can add them as comments.

    Increase LVM Volume size

    As part of a piece of capacity work that was actioned, we need to "add" a 30G LUN to increase the size of an existing file system to facilitate extra data we had predicted would be coming though as part of a new project. It transpired the VMware guys, via a few Chinese whispers, mistakenly "increased" the existing LUN size by 30G instead. We decided to leave this as it was, and dynamically try to expand the volume on-line. We have tried to shrink volumes before which resulted in loosing the volume and giving us a good test case to verify our restore process.

    Articles

    Java

    This what I have spent most of my time working in over the years and these are the little bits of logic or snippets I have come across. Some of these are things I have needed to search for or I have found I needed to feedback on code reviews.

    Oracle WebCenter (FatWire)

    During my time working in consulting I was tasked many times with delivering FatWire sites. I took notes due to the lack of useful documentation at the time. These are the results of those notes made and shared.

    CSS HTML JS

    I believe the majority of time on a project is spent with the UI. Web is no exception. I have compiled a set of articles covering HTML, CSS and JS with the intention of taking the sting out of this work.

    DevOps

    A space dedicated to all those things you need to know beyond writing code. This covers Linux, Windows, Git and SVN among other hopefully useful nuggets.

    Security

    I picked up some pointers over the years consulting for government and banking sectors before entering a PCI DSS development environment. This section covers aspects like coding, testing, code review and best practice.

    Apps

    Apps is a section which offers access to bits and pieces I have put together over the last few years. This includes Random Password Generator and Base64 Encoder.